Max level shown:
12025-12-16T03:00:04.207ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 12 reason = Timeout
22025-12-16T03:00:04.208ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
32025-12-16T03:00:04.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
42025-12-16T03:00:04.210ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52025-12-16T03:00:04.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
62025-12-16T03:00:04.213ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
72025-12-16T03:00:04.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
82025-12-16T03:00:04.218ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
92025-12-16T03:00:04.218ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 10.405362ms iteration = 12 status = {"config_updated":false}
102025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 3 reason = Timeout
112025-12-16T03:00:09.167ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
122025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 25.57\u{b5}s iteration = 3 status = {"error":"no config"}
132025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 2 reason = Timeout
142025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
152025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 2 reason = Timeout
162025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 3 reason = Timeout
172025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
182025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
192025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
202025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
212025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 3 reason = Timeout
222025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
232025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 2 reason = Timeout
242025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 3 reason = Timeout
252025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
262025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 2 reason = Timeout
272025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 2 reason = Timeout
282025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 2 reason = Timeout
292025-12-16T03:00:09.167ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-12-16 02:50:09.167469468 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
302025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
312025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 3 reason = Timeout
322025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
332025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 3 reason = Timeout
342025-12-16T03:00:09.167ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
352025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 16.661\u{b5}s iteration = 3 status = {"error":"no config"}
362025-12-16T03:00:09.167ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
372025-12-16T03:00:09.167ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fddd842f-0dc2-4433-afc8-0d922c7d270d step component = ExternalNetworking step id = Ensure
382025-12-16T03:00:09.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
392025-12-16T03:00:09.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
402025-12-16T03:00:09.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
412025-12-16T03:00:09.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
422025-12-16T03:00:09.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
432025-12-16T03:00:09.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
442025-12-16T03:00:09.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
452025-12-16T03:00:09.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
462025-12-16T03:00:09.168ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1a45cd23-3d0a-43eb-adce-62c7b712baf1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1016, tv_nsec: 786037689 } })
472025-12-16T03:00:09.168ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("49a51a59-9919-48ae-8182-4070a8430160.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6af60386-8771-4401-9d4d-e93774e519f0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 1016, tv_nsec: 786083819 } })
482025-12-16T03:00:09.168ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
492025-12-16T03:00:09.168ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.363418ms iteration = 2 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
502025-12-16T03:00:09.168ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
512025-12-16T03:00:09.168ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.465379ms iteration = 2 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
522025-12-16T03:00:09.169ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
532025-12-16T03:00:09.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
542025-12-16T03:00:09.171ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
552025-12-16T03:00:09.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
562025-12-16T03:00:09.172ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
572025-12-16T03:00:09.173ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-12-16 02:50:09.167469468 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
582025-12-16T03:00:09.173ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.811863ms iteration = 2 status = {"expiration":"2025-12-16T02:50:09.167469468Z","pruned":[]}
592025-12-16T03:00:09.173ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
602025-12-16T03:00:09.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
612025-12-16T03:00:09.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
622025-12-16T03:00:09.176ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
632025-12-16T03:00:09.176ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
642025-12-16T03:00:09.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
652025-12-16T03:00:09.176ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1016, tv_nsec: 794171942 } })
662025-12-16T03:00:09.177ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
672025-12-16T03:00:09.178ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1016, tv_nsec: 796016855 } })
682025-12-16T03:00:09.178ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
692025-12-16T03:00:09.178ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1016, tv_nsec: 796716284 } })
702025-12-16T03:00:09.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
712025-12-16T03:00:09.179ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 11.946601ms iteration = 3 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
722025-12-16T03:00:09.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
732025-12-16T03:00:09.180ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
742025-12-16T03:00:09.181ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
752025-12-16T03:00:09.182ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
762025-12-16T03:00:09.182ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
772025-12-16T03:00:09.182ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
782025-12-16T03:00:09.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
792025-12-16T03:00:09.183ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
802025-12-16T03:00:09.183ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
812025-12-16T03:00:09.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
822025-12-16T03:00:09.184ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
832025-12-16T03:00:09.184ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
842025-12-16T03:00:09.185ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
852025-12-16T03:00:09.185ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
862025-12-16T03:00:09.185ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
872025-12-16T03:00:09.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
882025-12-16T03:00:09.185ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
892025-12-16T03:00:09.187ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
902025-12-16T03:00:09.188ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
912025-12-16T03:00:09.191ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
922025-12-16T03:00:09.192ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
932025-12-16T03:00:09.192ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
942025-12-16T03:00:09.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
952025-12-16T03:00:09.195ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
962025-12-16T03:00:09.195ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
972025-12-16T03:00:09.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
982025-12-16T03:00:09.196ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
992025-12-16T03:00:09.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1002025-12-16T03:00:09.196ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
1012025-12-16T03:00:09.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1022025-12-16T03:00:09.197ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1032025-12-16T03:00:09.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
1042025-12-16T03:00:09.199ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1052025-12-16T03:00:09.199ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
1062025-12-16T03:00:09.199ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
1072025-12-16T03:00:09.200ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1082025-12-16T03:00:09.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1092025-12-16T03:00:09.200ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
1102025-12-16T03:00:09.203ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
1112025-12-16T03:00:09.203ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1122025-12-16T03:00:09.203ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1132025-12-16T03:00:09.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
1142025-12-16T03:00:09.204ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
1152025-12-16T03:00:09.204ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1162025-12-16T03:00:09.207ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
1172025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 6 reason = Timeout
1182025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 7 reason = Timeout
1192025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
1202025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 13 reason = Timeout
1212025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1222025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
1232025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 7 reason = Timeout
1242025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
1252025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 2 reason = Timeout
1262025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 3 reason = Timeout
1272025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
1282025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
1292025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1302025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 3 reason = Timeout
1312025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1322025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 3 reason = Timeout
1332025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1342025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 4 reason = Timeout
1352025-12-16T03:00:09.208ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
1362025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 2 reason = Timeout
1372025-12-16T03:00:09.208ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
1382025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1392025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
1402025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 4 reason = Timeout
1412025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
1422025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 2 reason = Timeout
1432025-12-16T03:00:09.208ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
1442025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 3 reason = Timeout
1452025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1462025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 7 reason = Timeout
1472025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1482025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 3 reason = Timeout
1492025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1502025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 5 reason = Timeout
1512025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 3 reason = Timeout
1522025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1532025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1542025-12-16T03:00:09.209ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
1552025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
1562025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
1572025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
1582025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
1592025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
1602025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
1612025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1622025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
1632025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
1642025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1652025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
1662025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1672025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
1682025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
1692025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1702025-12-16T03:00:09.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1712025-12-16T03:00:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1722025-12-16T03:00:09.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1732025-12-16T03:00:09.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1742025-12-16T03:00:09.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1752025-12-16T03:00:09.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1762025-12-16T03:00:09.211ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1772025-12-16T03:00:09.211ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1782025-12-16T03:00:09.211ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1792025-12-16T03:00:09.212ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
1802025-12-16T03:00:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1812025-12-16T03:00:09.212ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1822025-12-16T03:00:09.212ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1832025-12-16T03:00:09.212ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
1842025-12-16T03:00:09.212ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
1852025-12-16T03:00:09.212ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1862025-12-16T03:00:09.212ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1872025-12-16T03:00:09.213ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1882025-12-16T03:00:09.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
1892025-12-16T03:00:09.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1902025-12-16T03:00:09.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1912025-12-16T03:00:09.214ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1922025-12-16T03:00:09.214ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
1932025-12-16T03:00:09.214ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
1942025-12-16T03:00:09.214ZINFOnexus (BackgroundTasks): found latest generation (first find) background_task = dns_config_external dns_group = external file = nexus/src/app/background/tasks/dns_config.rs:87 generation = 2
1952025-12-16T03:00:09.214ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 47.387151ms iteration = 2 status = {"generation":2}
1962025-12-16T03:00:09.214ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 4 reason = Dependency
1972025-12-16T03:00:09.215ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
1982025-12-16T03:00:09.216ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1992025-12-16T03:00:09.217ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 8.215524ms iteration = 7 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2002025-12-16T03:00:09.218ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
2012025-12-16T03:00:09.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
2022025-12-16T03:00:09.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
2032025-12-16T03:00:09.218ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2042025-12-16T03:00:09.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
2052025-12-16T03:00:09.218ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
2062025-12-16T03:00:09.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
2072025-12-16T03:00:09.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2082025-12-16T03:00:09.218ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2092025-12-16T03:00:09.219ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2102025-12-16T03:00:09.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
2112025-12-16T03:00:09.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2122025-12-16T03:00:09.219ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
2132025-12-16T03:00:09.219ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2142025-12-16T03:00:09.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2152025-12-16T03:00:09.220ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
2162025-12-16T03:00:09.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
2172025-12-16T03:00:09.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
2182025-12-16T03:00:09.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2192025-12-16T03:00:09.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2202025-12-16T03:00:09.221ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2212025-12-16T03:00:09.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
2222025-12-16T03:00:09.221ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
2232025-12-16T03:00:09.221ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2242025-12-16T03:00:09.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
2252025-12-16T03:00:09.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2262025-12-16T03:00:09.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2272025-12-16T03:00:09.223ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
2282025-12-16T03:00:09.223ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2292025-12-16T03:00:09.223ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
2302025-12-16T03:00:09.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
2312025-12-16T03:00:09.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2322025-12-16T03:00:09.225ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
2332025-12-16T03:00:09.225ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 16.311477ms iteration = 13 status = {"config_updated":false}
2342025-12-16T03:00:09.225ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1016, tv_nsec: 843549788 } })
2352025-12-16T03:00:09.225ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 17.039666ms iteration = 3 status = 0
2362025-12-16T03:00:09.225ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 6 reason = Timeout
2372025-12-16T03:00:09.225ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
2382025-12-16T03:00:09.225ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 3 reason = Timeout
2392025-12-16T03:00:09.225ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 5 reason = Timeout
2402025-12-16T03:00:09.225ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 14.59\u{b5}s iteration = 6 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
2412025-12-16T03:00:09.225ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2422025-12-16T03:00:09.225ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
2432025-12-16T03:00:09.225ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 5 reason = Timeout
2442025-12-16T03:00:09.225ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2452025-12-16T03:00:09.225ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
2462025-12-16T03:00:09.225ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2472025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 3 reason = Timeout
2482025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 3 reason = Timeout
2492025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
2502025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2512025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
2522025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
2532025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 2 reason = Timeout
2542025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
2552025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 3 reason = Timeout
2562025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
2572025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 2 reason = Timeout
2582025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
2592025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 3 reason = Timeout
2602025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
2612025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 2 reason = Timeout
2622025-12-16T03:00:09.226ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
2632025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
2642025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
2652025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2662025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 3 reason = Timeout
2672025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
2682025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 8.11\u{b5}s iteration = 3 status = {"disabled":true,"errors":[],"sps":[]}
2692025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 2 reason = Timeout
2702025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
2712025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2722025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2732025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
2742025-12-16T03:00:09.226ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
2752025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
2762025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
2772025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2782025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
2792025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
2802025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
2812025-12-16T03:00:09.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2822025-12-16T03:00:09.226ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1016, tv_nsec: 844796233 } })
2832025-12-16T03:00:09.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
2842025-12-16T03:00:09.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
2852025-12-16T03:00:09.227ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
2862025-12-16T03:00:09.227ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2872025-12-16T03:00:09.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
2882025-12-16T03:00:09.227ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
2892025-12-16T03:00:09.232ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2902025-12-16T03:00:09.232ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
2912025-12-16T03:00:09.232ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2922025-12-16T03:00:09.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
2932025-12-16T03:00:09.236ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
2942025-12-16T03:00:09.236ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
2952025-12-16T03:00:09.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
2962025-12-16T03:00:09.238ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 12.176144ms iteration = 3 status = {"errors":[],"finish_invoked_ok":[]}
2972025-12-16T03:00:09.240ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
2982025-12-16T03:00:09.240ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
2992025-12-16T03:00:09.240ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 30.81487ms iteration = 2 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
3002025-12-16T03:00:09.240ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
3012025-12-16T03:00:09.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
3022025-12-16T03:00:09.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
3032025-12-16T03:00:09.248ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 21.830967ms iteration = 2 status = {"errors":[],"found_port_ok":[]}
3042025-12-16T03:00:09.249ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
3052025-12-16T03:00:09.249ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 22.919211ms iteration = 2 status = {"by_rx":{},"error":null}
3062025-12-16T03:00:09.249ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
3072025-12-16T03:00:09.249ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3082025-12-16T03:00:09.250ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fddd842f-0dc2-4433-afc8-0d922c7d270d step component = SledList step id = Fetch
3092025-12-16T03:00:09.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
3102025-12-16T03:00:09.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
3112025-12-16T03:00:09.254ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3122025-12-16T03:00:09.254ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
3132025-12-16T03:00:09.254ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
3142025-12-16T03:00:09.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
3152025-12-16T03:00:09.257ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
3162025-12-16T03:00:09.257ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
3172025-12-16T03:00:09.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
3182025-12-16T03:00:09.258ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3192025-12-16T03:00:09.262ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
3202025-12-16T03:00:09.262ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
3212025-12-16T03:00:09.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
3222025-12-16T03:00:09.266ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3232025-12-16T03:00:09.268ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
3242025-12-16T03:00:09.269ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 43.745294ms iteration = 5 status = "NoSitrep"
3252025-12-16T03:00:09.271ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
3262025-12-16T03:00:09.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
3272025-12-16T03:00:09.272ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3282025-12-16T03:00:09.276ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
3292025-12-16T03:00:09.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
3302025-12-16T03:00:09.277ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
3312025-12-16T03:00:09.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
3322025-12-16T03:00:09.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
3332025-12-16T03:00:09.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3342025-12-16T03:00:09.279ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
3352025-12-16T03:00:09.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
3362025-12-16T03:00:09.282ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
3372025-12-16T03:00:09.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
3382025-12-16T03:00:09.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
3392025-12-16T03:00:09.298ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
3402025-12-16T03:00:09.298ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
3412025-12-16T03:00:09.298ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3422025-12-16T03:00:09.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3432025-12-16T03:00:09.310ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3442025-12-16T03:00:09.310ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
3452025-12-16T03:00:09.310ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 102.077784ms iteration = 7 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
3462025-12-16T03:00:09.313ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
3472025-12-16T03:00:09.313ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
3482025-12-16T03:00:09.313ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
3492025-12-16T03:00:09.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
3502025-12-16T03:00:09.315ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
3512025-12-16T03:00:09.315ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
3522025-12-16T03:00:09.315ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
3532025-12-16T03:00:09.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
3542025-12-16T03:00:09.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
3552025-12-16T03:00:09.316ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3562025-12-16T03:00:09.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3572025-12-16T03:00:09.317ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
3582025-12-16T03:00:09.317ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
3592025-12-16T03:00:09.317ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 108.258951ms iteration = 2 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
3602025-12-16T03:00:09.317ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3612025-12-16T03:00:09.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3622025-12-16T03:00:09.319ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
3632025-12-16T03:00:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
3642025-12-16T03:00:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
3652025-12-16T03:00:09.320ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3662025-12-16T03:00:09.320ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 2ff2c680-3e77-43d1-87e0-51edcb3fda30 original_time_started = 2025-12-16 02:59:20.453839 UTC
3672025-12-16T03:00:09.320ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 112.00838ms iteration = 7 status = {"Loaded":{"collection_id":"2ff2c680-3e77-43d1-87e0-51edcb3fda30","time_loaded":"2025-12-16T03:00:09.208740631Z","time_started":"2025-12-16T02:59:20.453839Z"}}
3682025-12-16T03:00:09.320ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
3692025-12-16T03:00:09.320ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 94.874172ms iteration = 5 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
3702025-12-16T03:00:09.321ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
3712025-12-16T03:00:09.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
3722025-12-16T03:00:09.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
3732025-12-16T03:00:09.321ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3742025-12-16T03:00:09.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
3752025-12-16T03:00:09.323ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3762025-12-16T03:00:09.323ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 96.747156ms iteration = 2 status = {"dispatched":[],"errors":[],"glob_version":"212.0.0","globs_reprocessed":{},"no_receivers":[]}
3772025-12-16T03:00:09.324ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
3782025-12-16T03:00:09.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
3792025-12-16T03:00:09.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
3802025-12-16T03:00:09.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
3812025-12-16T03:00:09.326ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
3822025-12-16T03:00:09.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
3832025-12-16T03:00:09.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
3842025-12-16T03:00:09.326ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3852025-12-16T03:00:09.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
3862025-12-16T03:00:09.326ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
3872025-12-16T03:00:09.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3882025-12-16T03:00:09.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
3892025-12-16T03:00:09.333ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
3902025-12-16T03:00:09.333ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 124.737291ms iteration = 4 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
3912025-12-16T03:00:09.340ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
3922025-12-16T03:00:09.340ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
3932025-12-16T03:00:09.340ZINFOnexus (BackgroundTasks): found latest generation (first find) background_task = dns_config_internal dns_group = internal file = nexus/src/app/background/tasks/dns_config.rs:87 generation = 1
3942025-12-16T03:00:09.340ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 173.429538ms iteration = 2 status = {"generation":1}
3952025-12-16T03:00:09.340ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 4 reason = Dependency
3962025-12-16T03:00:09.343ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3972025-12-16T03:00:09.344ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 117.926294ms iteration = 3 status = {"errors":[],"requests_created_ok":[]}
3982025-12-16T03:00:09.345ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
3992025-12-16T03:00:09.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
4002025-12-16T03:00:09.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4012025-12-16T03:00:09.349ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
4022025-12-16T03:00:09.350ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 124.76863ms iteration = 3 status = {"errors":[],"garbage_collect_requested":[]}
4032025-12-16T03:00:09.351ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
4042025-12-16T03:00:09.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
4052025-12-16T03:00:09.375ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (initial) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-12-16T02:59:05.020237Z, time_modified: 2025-12-16T02:59:05.020237Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:77
4062025-12-16T03:00:09.375ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 208.449442ms iteration = 2 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]}},"default_endpoint":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
4072025-12-16T03:00:09.386ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 177.52561ms iteration = 3 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
4082025-12-16T03:00:09.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4092025-12-16T03:00:09.407ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4102025-12-16T03:00:09.407ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
4112025-12-16T03:00:09.407ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 199.009831ms iteration = 3 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
4122025-12-16T03:00:09.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4132025-12-16T03:00:09.410ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4142025-12-16T03:00:09.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4152025-12-16T03:00:09.412ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4162025-12-16T03:00:09.413ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
4172025-12-16T03:00:09.413ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4182025-12-16T03:00:09.413ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2350a724-4ece-4b3a-9b80-c716f89d9444", "content-length": "26", "date": "Tue, 16 Dec 2025 03:00:09 GMT"} })
4192025-12-16T03:00:09.414ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4202025-12-16T03:00:09.417ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
4212025-12-16T03:00:09.417ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4222025-12-16T03:00:09.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4232025-12-16T03:00:09.418ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
4242025-12-16T03:00:09.421ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
4252025-12-16T03:00:09.421ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 212.960429ms iteration = 3 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
4262025-12-16T03:00:09.421ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4272025-12-16T03:00:09.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4282025-12-16T03:00:09.422ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4292025-12-16T03:00:09.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
4302025-12-16T03:00:09.424ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4312025-12-16T03:00:09.425ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4322025-12-16T03:00:09.426ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 200.119595ms iteration = 3 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
4332025-12-16T03:00:09.429ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4342025-12-16T03:00:09.429ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4352025-12-16T03:00:09.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
4362025-12-16T03:00:09.430ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
4372025-12-16T03:00:09.430ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4382025-12-16T03:00:09.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4392025-12-16T03:00:09.431ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4402025-12-16T03:00:09.432ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4412025-12-16T03:00:09.434ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
4422025-12-16T03:00:09.434ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
4432025-12-16T03:00:09.434ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4442025-12-16T03:00:09.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
4452025-12-16T03:00:09.437ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 270.122963ms iteration = 3 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
4462025-12-16T03:00:09.437ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
4472025-12-16T03:00:09.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4482025-12-16T03:00:09.437ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 211.969816ms iteration = 3 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
4492025-12-16T03:00:09.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4502025-12-16T03:00:09.441ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
4512025-12-16T03:00:09.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
4522025-12-16T03:00:09.441ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4532025-12-16T03:00:09.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4542025-12-16T03:00:09.443ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
4552025-12-16T03:00:09.443ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4562025-12-16T03:00:09.443ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "da8fd488-9058-494a-82f8-446d6face4eb", "content-length": "26", "date": "Tue, 16 Dec 2025 03:00:09 GMT"} })
4572025-12-16T03:00:09.443ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4582025-12-16T03:00:09.443ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4592025-12-16T03:00:09.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4602025-12-16T03:00:09.447ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
4612025-12-16T03:00:09.447ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
4622025-12-16T03:00:09.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
4632025-12-16T03:00:09.448ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
4642025-12-16T03:00:09.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4652025-12-16T03:00:09.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4662025-12-16T03:00:09.449ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
4672025-12-16T03:00:09.450ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1017, tv_nsec: 68391406 } })
4682025-12-16T03:00:09.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4692025-12-16T03:00:09.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4702025-12-16T03:00:09.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4712025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = / versions = all
4722025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /access versions = all
4732025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /assets/{path} versions = all
4742025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /device/auth versions = all
4752025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /device/confirm versions = all
4762025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /device/success versions = all
4772025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /device/token versions = all
4782025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /device/verify versions = all
4792025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/probes versions = all
4802025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /experimental/v1/probes versions = all
4812025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /experimental/v1/probes/{probe} versions = all
4822025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/probes/{probe} versions = all
4832025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/system/support-bundles versions = all
4842025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /experimental/v1/system/support-bundles versions = all
4852025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
4862025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
4872025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
4882025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
4892025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
4902025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
4912025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
4922025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/index versions = all
4932025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /images versions = all
4942025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /login versions = all
4952025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /login/{silo_name}/local versions = all
4962025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
4972025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
4982025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
4992025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /lookup/{path} versions = all
5002025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /projects/{path} versions = all
5012025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /projects-new versions = all
5022025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /scim/v2/Groups versions = all
5032025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /scim/v2/Groups versions = all
5042025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /scim/v2/Groups/{group_id} versions = all
5052025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /scim/v2/Groups/{group_id} versions = all
5062025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PATCH path = /scim/v2/Groups/{group_id} versions = all
5072025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /scim/v2/Groups/{group_id} versions = all
5082025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /scim/v2/Users versions = all
5092025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /scim/v2/Users versions = all
5102025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /scim/v2/Users/{user_id} versions = all
5112025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /scim/v2/Users/{user_id} versions = all
5122025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PATCH path = /scim/v2/Users/{user_id} versions = all
5132025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /scim/v2/Users/{user_id} versions = all
5142025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /settings/{path} versions = all
5152025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /system/{path} versions = all
5162025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /utilization versions = all
5172025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/affinity-groups versions = all
5182025-12-16T03:00:09.459ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/affinity-groups versions = all
5192025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/affinity-groups/{affinity_group} versions = all
5202025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/affinity-groups/{affinity_group} versions = all
5212025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/affinity-groups/{affinity_group} versions = all
5222025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/affinity-groups/{affinity_group}/members versions = all
5232025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
5242025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
5252025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
5262025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/alert-classes versions = all
5272025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/alert-receivers versions = all
5282025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/alert-receivers/{receiver} versions = all
5292025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/alert-receivers/{receiver} versions = all
5302025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/alert-receivers/{receiver}/deliveries versions = all
5312025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/alert-receivers/{receiver}/probe versions = all
5322025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/alert-receivers/{receiver}/subscriptions versions = all
5332025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/alert-receivers/{receiver}/subscriptions/{subscription} versions = all
5342025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/alerts/{alert_id}/resend versions = all
5352025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/anti-affinity-groups versions = all
5362025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/anti-affinity-groups versions = all
5372025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
5382025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
5392025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
5402025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members versions = all
5412025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
5422025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
5432025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
5442025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/auth-settings versions = all
5452025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/auth-settings versions = all
5462025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/certificates versions = all
5472025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/certificates versions = all
5482025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/certificates/{certificate} versions = all
5492025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/certificates/{certificate} versions = all
5502025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/disks versions = all ending with 2025120300.0.0
5512025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/disks versions = all starting from 2025120300.0.0
5522025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/disks versions = all ending with 2025120300.0.0
5532025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/disks versions = all starting from 2025120300.0.0
5542025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/disks/{disk} versions = all
5552025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/disks/{disk} versions = all ending with 2025120300.0.0
5562025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/disks/{disk} versions = all starting from 2025120300.0.0
5572025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/disks/{disk}/bulk-write versions = all
5582025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
5592025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
5602025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/disks/{disk}/finalize versions = all
5612025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/floating-ips versions = all
5622025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/floating-ips versions = all
5632025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
5642025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/floating-ips/{floating_ip} versions = all
5652025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
5662025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
5672025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
5682025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/groups versions = all
5692025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/groups/{group_id} versions = all
5702025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/images versions = all
5712025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/images versions = all
5722025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/images/{image} versions = all
5732025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/images/{image} versions = all
5742025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/images/{image}/demote versions = all
5752025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/images/{image}/promote versions = all
5762025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances versions = all
5772025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances versions = all ending with 2025120300.0.0
5782025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances versions = all starting from 2025120300.0.0
5792025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/instances/{instance} versions = all
5802025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance} versions = all
5812025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/instances/{instance} versions = all
5822025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/affinity-groups versions = all
5832025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/anti-affinity-groups versions = all
5842025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/disks versions = all ending with 2025120300.0.0
5852025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/disks versions = all starting from 2025120300.0.0
5862025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/disks/attach versions = all ending with 2025120300.0.0
5872025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/disks/attach versions = all starting from 2025120300.0.0
5882025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/disks/detach versions = all ending with 2025120300.0.0
5892025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/disks/detach versions = all starting from 2025120300.0.0
5902025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/external-ips versions = all
5912025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
5922025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
5932025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/multicast-groups versions = all
5942025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/instances/{instance}/multicast-groups/{multicast_group} versions = all
5952025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/instances/{instance}/multicast-groups/{multicast_group} versions = all
5962025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/reboot versions = all
5972025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/serial-console versions = all
5982025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
5992025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
6002025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/start versions = all
6012025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/stop versions = all
6022025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/internet-gateway-ip-addresses versions = all
6032025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/internet-gateway-ip-addresses versions = all
6042025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
6052025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/internet-gateway-ip-pools versions = all
6062025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/internet-gateway-ip-pools versions = all
6072025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
6082025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/internet-gateways versions = all
6092025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/internet-gateways versions = all
6102025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
6112025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/internet-gateways/{gateway} versions = all
6122025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/ip-pools versions = all
6132025-12-16T03:00:09.460ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/ip-pools/{pool} versions = all
6142025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/login/{silo_name}/local versions = all
6152025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/logout versions = all
6162025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/me versions = all
6172025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/me/access-tokens versions = all
6182025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/me/access-tokens/{token_id} versions = all
6192025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/me/groups versions = all
6202025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/me/ssh-keys versions = all
6212025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/me/ssh-keys versions = all
6222025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
6232025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
6242025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/metrics/{metric_name} versions = all
6252025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/multicast-groups versions = all
6262025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/multicast-groups versions = all
6272025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/multicast-groups/{multicast_group} versions = all
6282025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/multicast-groups/{multicast_group} versions = all
6292025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/multicast-groups/{multicast_group} versions = all
6302025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/multicast-groups/{multicast_group}/members versions = all
6312025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/multicast-groups/{multicast_group}/members versions = all
6322025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/multicast-groups/{multicast_group}/members/{instance} versions = all
6332025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/network-interfaces versions = all
6342025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/network-interfaces versions = all
6352025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/network-interfaces/{interface} versions = all
6362025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/network-interfaces/{interface} versions = all
6372025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/network-interfaces/{interface} versions = all
6382025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/ping versions = all
6392025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/policy versions = all
6402025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/policy versions = all
6412025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/projects versions = all
6422025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/projects versions = all
6432025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/projects/{project} versions = all
6442025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/projects/{project} versions = all
6452025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/projects/{project} versions = all
6462025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/projects/{project}/policy versions = all
6472025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/projects/{project}/policy versions = all
6482025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/snapshots versions = all
6492025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/snapshots versions = all
6502025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/snapshots/{snapshot} versions = all
6512025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/snapshots/{snapshot} versions = all
6522025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/audit-log versions = all
6532025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/disks versions = all
6542025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
6552025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/rack-switch-port/{rack_id}/{switch_location}/{port}/lldp/neighbors versions = all
6562025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/racks versions = all
6572025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
6582025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/sleds versions = all
6592025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/hardware/sleds versions = all
6602025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
6612025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
6622025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
6632025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
6642025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
6652025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/switch-port versions = all
6662025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
6672025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
6682025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
6692025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
6702025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
6712025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/switches versions = all
6722025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
6732025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/identity-providers versions = all
6742025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/identity-providers/local/users versions = all
6752025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
6762025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
6772025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/identity-providers/saml versions = all
6782025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
6792025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools versions = all
6802025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools versions = all
6812025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
6822025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools/{pool} versions = all
6832025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/ip-pools/{pool} versions = all
6842025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
6852025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
6862025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
6872025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
6882025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
6892025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
6902025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
6912025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
6922025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools-service versions = all
6932025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools-service/ranges versions = all
6942025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
6952025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
6962025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/metrics/{metric_name} versions = all
6972025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/multicast-groups/by-ip/{address} versions = all
6982025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/address-lot versions = all
6992025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/address-lot versions = all
7002025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
7012025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/address-lot/{address_lot} versions = all
7022025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
7032025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/allow-list versions = all
7042025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/networking/allow-list versions = all
7052025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/bfd-disable versions = all
7062025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/bfd-enable versions = all
7072025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bfd-status versions = all
7082025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/networking/bgp versions = all
7092025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp versions = all
7102025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/bgp versions = all
7112025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-announce-set versions = all
7122025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
7132025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
7142025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
7152025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-exported versions = all
7162025-12-16T03:00:09.461ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-message-history versions = all
7172025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
7182025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-status versions = all ending with 2025121200.0.0
7192025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-status versions = all starting from 2025121200.0.0
7202025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/inbound-icmp versions = all
7212025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/networking/inbound-icmp versions = all
7222025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/loopback-address versions = all
7232025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/loopback-address versions = all
7242025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
7252025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
7262025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/switch-port-settings versions = all
7272025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/switch-port-settings versions = all
7282025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
7292025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/policy versions = all
7302025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/policy versions = all
7312025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/scim/tokens versions = all
7322025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/scim/tokens versions = all
7332025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/scim/tokens/{token_id} versions = all
7342025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/scim/tokens/{token_id} versions = all
7352025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silo-quotas versions = all
7362025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silos versions = all
7372025-12-16T03:00:09.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
7382025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/silos versions = all
7392025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/silos/{silo} versions = all
7402025-12-16T03:00:09.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
7412025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silos/{silo} versions = all
7422025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
7432025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silos/{silo}/policy versions = all
7442025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/silos/{silo}/policy versions = all
7452025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silos/{silo}/quotas versions = all
7462025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
7472025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/timeseries/query versions = all
7482025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/timeseries/schemas versions = all
7492025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/update/repositories versions = all
7502025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/update/repositories versions = all
7512025-12-16T03:00:09.462ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/update/repositories/{system_version} versions = all
7522025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/update/status versions = all
7532025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/update/target-release versions = all
7542025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/update/trust-roots versions = all
7552025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/update/trust-roots versions = all
7562025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/update/trust-roots/{trust_root_id} versions = all
7572025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/update/trust-roots/{trust_root_id} versions = all
7582025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/users versions = all
7592025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/users/{user_id} versions = all
7602025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/users-builtin versions = all
7612025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/users-builtin/{user} versions = all
7622025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/utilization/silos versions = all
7632025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/utilization/silos/{silo} versions = all
7642025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/timeseries/query versions = all
7652025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/users versions = all
7662025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/users/{user_id} versions = all
7672025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/users/{user_id}/access-tokens versions = all
7682025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/users/{user_id}/logout versions = all
7692025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/users/{user_id}/sessions versions = all
7702025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/utilization versions = all
7712025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-firewall-rules versions = all
7722025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/vpc-firewall-rules versions = all
7732025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-router-routes versions = all
7742025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/vpc-router-routes versions = all
7752025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
7762025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-router-routes/{route} versions = all
7772025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/vpc-router-routes/{route} versions = all
7782025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-routers versions = all
7792025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/vpc-routers versions = all
7802025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/vpc-routers/{router} versions = all
7812025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-routers/{router} versions = all
7822025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/vpc-routers/{router} versions = all
7832025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-subnets versions = all
7842025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/vpc-subnets versions = all
7852025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
7862025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-subnets/{subnet} versions = all
7872025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
7882025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
7892025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpcs versions = all
7902025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/vpcs versions = all
7912025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/vpcs/{vpc} versions = all
7922025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpcs/{vpc} versions = all
7932025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/vpcs/{vpc} versions = all
7942025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/webhook-receivers versions = all
7952025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/webhook-receivers/{receiver} versions = all
7962025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/webhook-secrets versions = all
7972025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/webhook-secrets versions = all
7982025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/webhook-secrets/{secret_id} versions = all
7992025-12-16T03:00:09.463ZINFOnexus (dropshot_external): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:328 local_addr = 172.30.2.5:443
8002025-12-16T03:00:09.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
8012025-12-16T03:00:09.463ZDEBGnexus (dropshot_external): successfully registered DTrace USDT probes local_addr = 172.30.2.5:443
8022025-12-16T03:00:09.468ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
8032025-12-16T03:00:09.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
8042025-12-16T03:00:09.472ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8052025-12-16T03:00:09.477ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
8062025-12-16T03:00:09.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
8072025-12-16T03:00:09.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
8082025-12-16T03:00:09.478ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = / versions = all
8092025-12-16T03:00:09.478ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /access versions = all
8102025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /assets/{path} versions = all
8112025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /device/auth versions = all
8122025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /device/confirm versions = all
8132025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /device/success versions = all
8142025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /device/token versions = all
8152025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /device/verify versions = all
8162025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/probes versions = all
8172025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /experimental/v1/probes versions = all
8182025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /experimental/v1/probes/{probe} versions = all
8192025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/probes/{probe} versions = all
8202025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/system/support-bundles versions = all
8212025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /experimental/v1/system/support-bundles versions = all
8222025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
8232025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
8242025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
8252025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
8262025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
8272025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
8282025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
8292025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/index versions = all
8302025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /images versions = all
8312025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /login versions = all
8322025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /login/{silo_name}/local versions = all
8332025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
8342025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
8352025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
8362025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /lookup/{path} versions = all
8372025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /projects/{path} versions = all
8382025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /projects-new versions = all
8392025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /scim/v2/Groups versions = all
8402025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /scim/v2/Groups versions = all
8412025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /scim/v2/Groups/{group_id} versions = all
8422025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /scim/v2/Groups/{group_id} versions = all
8432025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PATCH path = /scim/v2/Groups/{group_id} versions = all
8442025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /scim/v2/Groups/{group_id} versions = all
8452025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /scim/v2/Users versions = all
8462025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /scim/v2/Users versions = all
8472025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /scim/v2/Users/{user_id} versions = all
8482025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /scim/v2/Users/{user_id} versions = all
8492025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PATCH path = /scim/v2/Users/{user_id} versions = all
8502025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /scim/v2/Users/{user_id} versions = all
8512025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /settings/{path} versions = all
8522025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /system/{path} versions = all
8532025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /utilization versions = all
8542025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/affinity-groups versions = all
8552025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/affinity-groups versions = all
8562025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/affinity-groups/{affinity_group} versions = all
8572025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/affinity-groups/{affinity_group} versions = all
8582025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/affinity-groups/{affinity_group} versions = all
8592025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/affinity-groups/{affinity_group}/members versions = all
8602025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
8612025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
8622025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
8632025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/alert-classes versions = all
8642025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/alert-receivers versions = all
8652025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/alert-receivers/{receiver} versions = all
8662025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/alert-receivers/{receiver} versions = all
8672025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/alert-receivers/{receiver}/deliveries versions = all
8682025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/alert-receivers/{receiver}/probe versions = all
8692025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/alert-receivers/{receiver}/subscriptions versions = all
8702025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/alert-receivers/{receiver}/subscriptions/{subscription} versions = all
8712025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/alerts/{alert_id}/resend versions = all
8722025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/anti-affinity-groups versions = all
8732025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/anti-affinity-groups versions = all
8742025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
8752025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
8762025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
8772025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members versions = all
8782025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
8792025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
8802025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
8812025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/auth-settings versions = all
8822025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/auth-settings versions = all
8832025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/certificates versions = all
8842025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/certificates versions = all
8852025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/certificates/{certificate} versions = all
8862025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/certificates/{certificate} versions = all
8872025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/disks versions = all ending with 2025120300.0.0
8882025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/disks versions = all starting from 2025120300.0.0
8892025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/disks versions = all ending with 2025120300.0.0
8902025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/disks versions = all starting from 2025120300.0.0
8912025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/disks/{disk} versions = all
8922025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/disks/{disk} versions = all ending with 2025120300.0.0
8932025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/disks/{disk} versions = all starting from 2025120300.0.0
8942025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/disks/{disk}/bulk-write versions = all
8952025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
8962025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
8972025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/disks/{disk}/finalize versions = all
8982025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/floating-ips versions = all
8992025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/floating-ips versions = all
9002025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
9012025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/floating-ips/{floating_ip} versions = all
9022025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
9032025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
9042025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
9052025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/groups versions = all
9062025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/groups/{group_id} versions = all
9072025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/images versions = all
9082025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/images versions = all
9092025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/images/{image} versions = all
9102025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/images/{image} versions = all
9112025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/images/{image}/demote versions = all
9122025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/images/{image}/promote versions = all
9132025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances versions = all
9142025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances versions = all ending with 2025120300.0.0
9152025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances versions = all starting from 2025120300.0.0
9162025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/instances/{instance} versions = all
9172025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance} versions = all
9182025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/instances/{instance} versions = all
9192025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/affinity-groups versions = all
9202025-12-16T03:00:09.479ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/anti-affinity-groups versions = all
9212025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/disks versions = all ending with 2025120300.0.0
9222025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/disks versions = all starting from 2025120300.0.0
9232025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/disks/attach versions = all ending with 2025120300.0.0
9242025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/disks/attach versions = all starting from 2025120300.0.0
9252025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/disks/detach versions = all ending with 2025120300.0.0
9262025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/disks/detach versions = all starting from 2025120300.0.0
9272025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/external-ips versions = all
9282025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
9292025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
9302025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/multicast-groups versions = all
9312025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/instances/{instance}/multicast-groups/{multicast_group} versions = all
9322025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/instances/{instance}/multicast-groups/{multicast_group} versions = all
9332025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/reboot versions = all
9342025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/serial-console versions = all
9352025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
9362025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
9372025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/start versions = all
9382025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/stop versions = all
9392025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/internet-gateway-ip-addresses versions = all
9402025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/internet-gateway-ip-addresses versions = all
9412025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
9422025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/internet-gateway-ip-pools versions = all
9432025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/internet-gateway-ip-pools versions = all
9442025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
9452025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/internet-gateways versions = all
9462025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/internet-gateways versions = all
9472025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
9482025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/internet-gateways/{gateway} versions = all
9492025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/ip-pools versions = all
9502025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/ip-pools/{pool} versions = all
9512025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/login/{silo_name}/local versions = all
9522025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/logout versions = all
9532025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/me versions = all
9542025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/me/access-tokens versions = all
9552025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/me/access-tokens/{token_id} versions = all
9562025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/me/groups versions = all
9572025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/me/ssh-keys versions = all
9582025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/me/ssh-keys versions = all
9592025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
9602025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
9612025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/metrics/{metric_name} versions = all
9622025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/multicast-groups versions = all
9632025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/multicast-groups versions = all
9642025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/multicast-groups/{multicast_group} versions = all
9652025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/multicast-groups/{multicast_group} versions = all
9662025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/multicast-groups/{multicast_group} versions = all
9672025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/multicast-groups/{multicast_group}/members versions = all
9682025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/multicast-groups/{multicast_group}/members versions = all
9692025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/multicast-groups/{multicast_group}/members/{instance} versions = all
9702025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/network-interfaces versions = all
9712025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/network-interfaces versions = all
9722025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/network-interfaces/{interface} versions = all
9732025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/network-interfaces/{interface} versions = all
9742025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/network-interfaces/{interface} versions = all
9752025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/ping versions = all
9762025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/policy versions = all
9772025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/policy versions = all
9782025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/projects versions = all
9792025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/projects versions = all
9802025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/projects/{project} versions = all
9812025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/projects/{project} versions = all
9822025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/projects/{project} versions = all
9832025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/projects/{project}/policy versions = all
9842025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/projects/{project}/policy versions = all
9852025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/snapshots versions = all
9862025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/snapshots versions = all
9872025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/snapshots/{snapshot} versions = all
9882025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/snapshots/{snapshot} versions = all
9892025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/audit-log versions = all
9902025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/disks versions = all
9912025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
9922025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/rack-switch-port/{rack_id}/{switch_location}/{port}/lldp/neighbors versions = all
9932025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/racks versions = all
9942025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
9952025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/sleds versions = all
9962025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/hardware/sleds versions = all
9972025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
9982025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
9992025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
10002025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
10012025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
10022025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/switch-port versions = all
10032025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
10042025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
10052025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
10062025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
10072025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
10082025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/switches versions = all
10092025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
10102025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/identity-providers versions = all
10112025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/identity-providers/local/users versions = all
10122025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
10132025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
10142025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/identity-providers/saml versions = all
10152025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
10162025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools versions = all
10172025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools versions = all
10182025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
10192025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools/{pool} versions = all
10202025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/ip-pools/{pool} versions = all
10212025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
10222025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
10232025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
10242025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
10252025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
10262025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
10272025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
10282025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
10292025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools-service versions = all
10302025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools-service/ranges versions = all
10312025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
10322025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
10332025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/metrics/{metric_name} versions = all
10342025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/multicast-groups/by-ip/{address} versions = all
10352025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/address-lot versions = all
10362025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/address-lot versions = all
10372025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
10382025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/address-lot/{address_lot} versions = all
10392025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
10402025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/allow-list versions = all
10412025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/networking/allow-list versions = all
10422025-12-16T03:00:09.480ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/bfd-disable versions = all
10432025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/bfd-enable versions = all
10442025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bfd-status versions = all
10452025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/networking/bgp versions = all
10462025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp versions = all
10472025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/bgp versions = all
10482025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-announce-set versions = all
10492025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
10502025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
10512025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
10522025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-exported versions = all
10532025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-message-history versions = all
10542025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
10552025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-status versions = all ending with 2025121200.0.0
10562025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-status versions = all starting from 2025121200.0.0
10572025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/inbound-icmp versions = all
10582025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/networking/inbound-icmp versions = all
10592025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/loopback-address versions = all
10602025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/loopback-address versions = all
10612025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
10622025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
10632025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/switch-port-settings versions = all
10642025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/switch-port-settings versions = all
10652025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
10662025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/policy versions = all
10672025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/policy versions = all
10682025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/scim/tokens versions = all
10692025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/scim/tokens versions = all
10702025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/scim/tokens/{token_id} versions = all
10712025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/scim/tokens/{token_id} versions = all
10722025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silo-quotas versions = all
10732025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silos versions = all
10742025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/silos versions = all
10752025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/silos/{silo} versions = all
10762025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silos/{silo} versions = all
10772025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
10782025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silos/{silo}/policy versions = all
10792025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/silos/{silo}/policy versions = all
10802025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silos/{silo}/quotas versions = all
10812025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
10822025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/timeseries/query versions = all
10832025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/timeseries/schemas versions = all
10842025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/update/repositories versions = all
10852025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/update/repositories versions = all
10862025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/update/repositories/{system_version} versions = all
10872025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/update/status versions = all
10882025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/update/target-release versions = all
10892025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/update/trust-roots versions = all
10902025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/update/trust-roots versions = all
10912025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/update/trust-roots/{trust_root_id} versions = all
10922025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/update/trust-roots/{trust_root_id} versions = all
10932025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/users versions = all
10942025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/users/{user_id} versions = all
10952025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/users-builtin versions = all
10962025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/users-builtin/{user} versions = all
10972025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/utilization/silos versions = all
10982025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/utilization/silos/{silo} versions = all
10992025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/timeseries/query versions = all
11002025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/users versions = all
11012025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/users/{user_id} versions = all
11022025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/users/{user_id}/access-tokens versions = all
11032025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/users/{user_id}/logout versions = all
11042025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/users/{user_id}/sessions versions = all
11052025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/utilization versions = all
11062025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-firewall-rules versions = all
11072025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/vpc-firewall-rules versions = all
11082025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-router-routes versions = all
11092025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/vpc-router-routes versions = all
11102025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
11112025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-router-routes/{route} versions = all
11122025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/vpc-router-routes/{route} versions = all
11132025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-routers versions = all
11142025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/vpc-routers versions = all
11152025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/vpc-routers/{router} versions = all
11162025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-routers/{router} versions = all
11172025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/vpc-routers/{router} versions = all
11182025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-subnets versions = all
11192025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/vpc-subnets versions = all
11202025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
11212025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-subnets/{subnet} versions = all
11222025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
11232025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
11242025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpcs versions = all
11252025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/vpcs versions = all
11262025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/vpcs/{vpc} versions = all
11272025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpcs/{vpc} versions = all
11282025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/vpcs/{vpc} versions = all
11292025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/webhook-receivers versions = all
11302025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/webhook-receivers/{receiver} versions = all
11312025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/webhook-secrets versions = all
11322025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/webhook-secrets versions = all
11332025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/webhook-secrets/{secret_id} versions = all
11342025-12-16T03:00:09.481ZINFOnexus (dropshot_external_techport): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:328 local_addr = [fd00:1122:3344:101::a]:12228
11352025-12-16T03:00:09.481ZDEBGnexus (dropshot_external_techport): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::a]:12228
11362025-12-16T03:00:09.486ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fddd842f-0dc2-4433-afc8-0d922c7d270d step component = DeployNexusRecords step id = Ensure
11372025-12-16T03:00:09.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
11382025-12-16T03:00:09.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
11392025-12-16T03:00:09.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11402025-12-16T03:00:09.491ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
11412025-12-16T03:00:09.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
11422025-12-16T03:00:09.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
11432025-12-16T03:00:09.492ZDEBGnexus (nexus-producer-server): registered DTrace probes
11442025-12-16T03:00:09.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
11452025-12-16T03:00:09.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
11462025-12-16T03:00:09.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11472025-12-16T03:00:09.500ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 291.441502ms iteration = 5 status = {"physical_disks_added":0}
11482025-12-16T03:00:09.500ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
11492025-12-16T03:00:09.505ZDEBGnexus (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::a]:54115 method = GET path = /{producer_id} versions = all
11502025-12-16T03:00:09.506ZINFOnexus (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:328 local_addr = [fd00:1122:3344:101::a]:54115
11512025-12-16T03:00:09.506ZDEBGnexus (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::a]:54115
11522025-12-16T03:00:09.506ZDEBGnexus (nexus-producer-server): Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::a]:54115
11532025-12-16T03:00:09.506ZDEBGnexus (nexus-producer-server): Nexus IP provided explicitly, will use it addr = [fd00:1122:3344:101::a]:12221
11542025-12-16T03:00:09.506ZDEBGnexus (nexus-producer-server): starting producer registration task
11552025-12-16T03:00:09.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11562025-12-16T03:00:09.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11572025-12-16T03:00:09.508ZINFOnexus (nexus-producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::a]:54115 file = oximeter/producer/src/lib.rs:279 interval = 10s producer_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
11582025-12-16T03:00:09.508ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0
11592025-12-16T03:00:09.508ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221
11602025-12-16T03:00:09.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11612025-12-16T03:00:09.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11622025-12-16T03:00:09.511ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
11632025-12-16T03:00:09.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11642025-12-16T03:00:09.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
11652025-12-16T03:00:09.516ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
11662025-12-16T03:00:09.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11672025-12-16T03:00:09.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11682025-12-16T03:00:09.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11692025-12-16T03:00:09.524ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
11702025-12-16T03:00:09.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11712025-12-16T03:00:09.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11722025-12-16T03:00:09.525ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
11732025-12-16T03:00:09.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11742025-12-16T03:00:09.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11752025-12-16T03:00:09.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11762025-12-16T03:00:09.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11772025-12-16T03:00:09.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11782025-12-16T03:00:09.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
11792025-12-16T03:00:09.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11802025-12-16T03:00:09.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11812025-12-16T03:00:09.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11822025-12-16T03:00:09.544ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
11832025-12-16T03:00:09.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11842025-12-16T03:00:09.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11852025-12-16T03:00:09.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11862025-12-16T03:00:09.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11872025-12-16T03:00:09.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11882025-12-16T03:00:09.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11892025-12-16T03:00:09.550ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11902025-12-16T03:00:09.553ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fddd842f-0dc2-4433-afc8-0d922c7d270d step component = SledAgent step id = Ensure
11912025-12-16T03:00:09.554ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
11922025-12-16T03:00:09.555ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11932025-12-16T03:00:09.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11942025-12-16T03:00:09.556ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11952025-12-16T03:00:09.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11962025-12-16T03:00:09.558ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11972025-12-16T03:00:09.562ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
11982025-12-16T03:00:09.563ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11992025-12-16T03:00:09.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12002025-12-16T03:00:09.570ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12012025-12-16T03:00:09.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12022025-12-16T03:00:09.572ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12032025-12-16T03:00:09.577ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
12042025-12-16T03:00:09.577ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12052025-12-16T03:00:09.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12062025-12-16T03:00:09.578ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12072025-12-16T03:00:09.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12082025-12-16T03:00:09.580ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12092025-12-16T03:00:09.584ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
12102025-12-16T03:00:09.584ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12112025-12-16T03:00:09.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12122025-12-16T03:00:09.588ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12132025-12-16T03:00:09.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12142025-12-16T03:00:09.589ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12152025-12-16T03:00:09.592ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
12162025-12-16T03:00:09.592ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12172025-12-16T03:00:09.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12182025-12-16T03:00:09.605ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12192025-12-16T03:00:09.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12202025-12-16T03:00:09.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12212025-12-16T03:00:09.610ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
12222025-12-16T03:00:09.610ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12232025-12-16T03:00:09.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12242025-12-16T03:00:09.648ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 481.183376ms iteration = 3 status = {}
12252025-12-16T03:00:09.674ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
12262025-12-16T03:00:09.674ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9c46548-f2e1-454c-adaf-d0e716f82f4c", "content-length": "691", "date": "Tue, 16 Dec 2025 03:00:09 GMT"} })
12272025-12-16T03:00:09.674ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
12282025-12-16T03:00:09.674ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
12292025-12-16T03:00:09.675ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
12302025-12-16T03:00:09.676ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 467.670855ms iteration = 3 status = {}
12312025-12-16T03:00:09.678ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1017, tv_nsec: 296170122 } })
12322025-12-16T03:00:09.681ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
12332025-12-16T03:00:09.681ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
12342025-12-16T03:00:09.681ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = a6e6f687-eb09-4ac0-9e08-887cf449a2c3 parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
12352025-12-16T03:00:09.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12362025-12-16T03:00:09.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12372025-12-16T03:00:09.683ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
12382025-12-16T03:00:09.683ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 474.240759ms iteration = 2 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
12392025-12-16T03:00:09.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12402025-12-16T03:00:09.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
12412025-12-16T03:00:09.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12422025-12-16T03:00:09.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12432025-12-16T03:00:09.690ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
12442025-12-16T03:00:09.690ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
12452025-12-16T03:00:09.690ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 481.698753ms iteration = 6 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
12462025-12-16T03:00:09.699ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
12472025-12-16T03:00:09.699ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "2333c076-6163-4a28-9ac4-917bd96a02ce", "date": "Tue, 16 Dec 2025 03:00:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
12482025-12-16T03:00:09.700ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
12492025-12-16T03:00:09.700ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "c06071d9-3562-492c-9ed9-bf1424ab5166", "date": "Tue, 16 Dec 2025 03:00:09 GMT"} })
12502025-12-16T03:00:09.700ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
12512025-12-16T03:00:09.702ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
12522025-12-16T03:00:09.704ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
12532025-12-16T03:00:09.704ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 477.929585ms iteration = 2 status = {"errors":[],"probes_by_sled":{"b1c76b61-acee-4be6-b582-973c938d6b6e":0}}
12542025-12-16T03:00:09.705ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "89eccc68-da02-4ef7-b7bb-988d52f5c31d", "date": "Tue, 16 Dec 2025 03:00:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
12552025-12-16T03:00:09.705ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
12562025-12-16T03:00:09.707ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
12572025-12-16T03:00:09.707ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 492.48554ms iteration = 4 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
12582025-12-16T03:00:09.760ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
12592025-12-16T03:00:09.768ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
12602025-12-16T03:00:09.770ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "04608b4d-049b-44e6-9f40-68200a76bf21", "date": "Tue, 16 Dec 2025 03:00:09 GMT"} }) sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
12612025-12-16T03:00:09.772ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fddd842f-0dc2-4433-afc8-0d922c7d270d step component = FirewallRules step id = Ensure
12622025-12-16T03:00:09.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12632025-12-16T03:00:09.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12642025-12-16T03:00:09.774ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12652025-12-16T03:00:09.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12662025-12-16T03:00:09.776ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12672025-12-16T03:00:09.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12682025-12-16T03:00:09.779ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12692025-12-16T03:00:09.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12702025-12-16T03:00:09.781ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12712025-12-16T03:00:09.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12722025-12-16T03:00:09.782ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12732025-12-16T03:00:09.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12742025-12-16T03:00:09.784ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12752025-12-16T03:00:09.789ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
12762025-12-16T03:00:09.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12772025-12-16T03:00:09.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12782025-12-16T03:00:09.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12792025-12-16T03:00:09.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12802025-12-16T03:00:09.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12812025-12-16T03:00:09.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12822025-12-16T03:00:09.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12832025-12-16T03:00:09.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
12842025-12-16T03:00:09.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12852025-12-16T03:00:09.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12862025-12-16T03:00:09.802ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12872025-12-16T03:00:09.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12882025-12-16T03:00:09.804ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12892025-12-16T03:00:09.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12902025-12-16T03:00:09.805ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12912025-12-16T03:00:09.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12922025-12-16T03:00:09.805ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
12932025-12-16T03:00:09.805ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
12942025-12-16T03:00:09.806ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a52699b-4547-4521-998e-ba75bf7f4eba", "content-length": "372", "date": "Tue, 16 Dec 2025 03:00:09 GMT"} })
12952025-12-16T03:00:09.806ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12962025-12-16T03:00:09.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12972025-12-16T03:00:09.807ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12982025-12-16T03:00:09.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12992025-12-16T03:00:09.808ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1017, tv_nsec: 426310381 } })
13002025-12-16T03:00:09.808ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13012025-12-16T03:00:09.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13022025-12-16T03:00:09.810ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13032025-12-16T03:00:09.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13042025-12-16T03:00:09.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13052025-12-16T03:00:09.814ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
13062025-12-16T03:00:09.814ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:49762
13072025-12-16T03:00:09.814ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:49762 req_id = 1d1d1337-a597-4f9c-9620-17d75b450830 uri = /metrics/producers version = 1.0.0
13082025-12-16T03:00:09.816ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
13092025-12-16T03:00:09.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13102025-12-16T03:00:09.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13112025-12-16T03:00:09.817ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13122025-12-16T03:00:09.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13132025-12-16T03:00:09.821ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13142025-12-16T03:00:09.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13152025-12-16T03:00:09.827ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13162025-12-16T03:00:09.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13172025-12-16T03:00:09.830ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13182025-12-16T03:00:09.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13192025-12-16T03:00:09.832ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13202025-12-16T03:00:09.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13212025-12-16T03:00:09.833ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:49762 req_id = 1d1d1337-a597-4f9c-9620-17d75b450830 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
13222025-12-16T03:00:09.834ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:49762 req_id = 1d1d1337-a597-4f9c-9620-17d75b450830 resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
13232025-12-16T03:00:09.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13242025-12-16T03:00:09.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13252025-12-16T03:00:09.842ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13262025-12-16T03:00:09.845ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
13272025-12-16T03:00:09.845ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "acde9bff-e36e-4373-8644-a74ed3b2af25", "date": "Tue, 16 Dec 2025 03:00:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
13282025-12-16T03:00:09.845ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
13292025-12-16T03:00:09.847ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "451a8370-b8f6-4517-95c3-e3c8950e3c98", "date": "Tue, 16 Dec 2025 03:00:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
13302025-12-16T03:00:09.847ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
13312025-12-16T03:00:09.847ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
13322025-12-16T03:00:09.847ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13332025-12-16T03:00:09.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13342025-12-16T03:00:09.849ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "f5313c70-9c0d-4a1b-be42-7fc7d4ea0c5f", "date": "Tue, 16 Dec 2025 03:00:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
13352025-12-16T03:00:09.849ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
13362025-12-16T03:00:09.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13372025-12-16T03:00:09.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13382025-12-16T03:00:09.851ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13392025-12-16T03:00:09.851ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
13402025-12-16T03:00:09.851ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 510.856843ms iteration = 4 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
13412025-12-16T03:00:09.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13422025-12-16T03:00:09.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13432025-12-16T03:00:09.858ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
13442025-12-16T03:00:09.858ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13452025-12-16T03:00:09.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13462025-12-16T03:00:09.859ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13472025-12-16T03:00:09.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13482025-12-16T03:00:09.862ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
13492025-12-16T03:00:09.862ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13502025-12-16T03:00:09.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13512025-12-16T03:00:09.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13522025-12-16T03:00:09.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13532025-12-16T03:00:09.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13542025-12-16T03:00:09.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13552025-12-16T03:00:09.868ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
13562025-12-16T03:00:09.869ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
13572025-12-16T03:00:09.869ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "ee3276b3-4615-4c58-b04d-7f3e346d06b6", "date": "Tue, 16 Dec 2025 03:00:09 GMT"} })
13582025-12-16T03:00:09.869ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13592025-12-16T03:00:09.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13602025-12-16T03:00:09.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13612025-12-16T03:00:09.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13622025-12-16T03:00:09.871ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13632025-12-16T03:00:09.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13642025-12-16T03:00:09.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13652025-12-16T03:00:09.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13662025-12-16T03:00:09.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13672025-12-16T03:00:09.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13682025-12-16T03:00:09.874ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13692025-12-16T03:00:09.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13702025-12-16T03:00:09.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13712025-12-16T03:00:09.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13722025-12-16T03:00:09.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13732025-12-16T03:00:09.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13742025-12-16T03:00:09.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13752025-12-16T03:00:09.882ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
13762025-12-16T03:00:09.882ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
13772025-12-16T03:00:09.883ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "57a4b7f7-7283-404f-b627-eee47890388c", "date": "Tue, 16 Dec 2025 03:00:09 GMT"} })
13782025-12-16T03:00:09.883ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
13792025-12-16T03:00:09.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13802025-12-16T03:00:09.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13812025-12-16T03:00:09.884ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 676.006695ms iteration = 4 status = {}
13822025-12-16T03:00:09.884ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 5 reason = Dependency
13832025-12-16T03:00:09.884ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
13842025-12-16T03:00:09.884ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13852025-12-16T03:00:09.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13862025-12-16T03:00:09.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13872025-12-16T03:00:09.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13882025-12-16T03:00:09.886ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13892025-12-16T03:00:09.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13902025-12-16T03:00:09.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13912025-12-16T03:00:09.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13922025-12-16T03:00:09.888ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
13932025-12-16T03:00:09.888ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13942025-12-16T03:00:09.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13952025-12-16T03:00:09.890ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13962025-12-16T03:00:09.893ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
13972025-12-16T03:00:09.893ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13982025-12-16T03:00:09.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13992025-12-16T03:00:09.894ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
14002025-12-16T03:00:09.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14012025-12-16T03:00:09.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14022025-12-16T03:00:09.904ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }], Name("external-dns"): [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
14032025-12-16T03:00:09.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14042025-12-16T03:00:09.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14052025-12-16T03:00:09.905ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14062025-12-16T03:00:09.908ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
14072025-12-16T03:00:09.908ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14082025-12-16T03:00:09.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14092025-12-16T03:00:09.908ZINFOnexus: assigned collector to new producer collector_id = e5b81e4f-90b9-43ba-ba56-584fd121e601 file = nexus/src/app/oximeter.rs:76 producer_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
14102025-12-16T03:00:09.909ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 94341 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:49762 req_id = 1d1d1337-a597-4f9c-9620-17d75b450830 response_code = 201 uri = /metrics/producers
14112025-12-16T03:00:09.909ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "1d1d1337-a597-4f9c-9620-17d75b450830", "content-length": "41", "date": "Tue, 16 Dec 2025 03:00:09 GMT"} })
14122025-12-16T03:00:09.909ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
14132025-12-16T03:00:09.909ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
14142025-12-16T03:00:09.910ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
14152025-12-16T03:00:09.910ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
14162025-12-16T03:00:09.929ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
14172025-12-16T03:00:09.929ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
14182025-12-16T03:00:09.930ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a46fe92-3025-4735-8aa5-f923cce08cce", "content-length": "372", "date": "Tue, 16 Dec 2025 03:00:09 GMT"} })
14192025-12-16T03:00:09.930ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14202025-12-16T03:00:09.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14212025-12-16T03:00:09.944ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
14222025-12-16T03:00:09.944ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
14232025-12-16T03:00:09.945ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "677d7361-0040-445e-80d6-7afb2f6f1fa2", "date": "Tue, 16 Dec 2025 03:00:09 GMT"} })
14242025-12-16T03:00:09.945ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14252025-12-16T03:00:09.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14262025-12-16T03:00:09.947ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14272025-12-16T03:00:09.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14282025-12-16T03:00:09.951ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
14292025-12-16T03:00:09.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
14302025-12-16T03:00:09.952ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
14312025-12-16T03:00:09.952ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
14322025-12-16T03:00:09.953ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
14332025-12-16T03:00:09.953ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
14342025-12-16T03:00:09.953ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
14352025-12-16T03:00:09.953ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
14362025-12-16T03:00:09.953ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
14372025-12-16T03:00:09.953ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
14382025-12-16T03:00:09.953ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
14392025-12-16T03:00:09.953ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
14402025-12-16T03:00:09.954ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
14412025-12-16T03:00:09.954ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14422025-12-16T03:00:09.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14432025-12-16T03:00:09.954ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
14442025-12-16T03:00:09.954ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
14452025-12-16T03:00:09.955ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "b864ae93-31d7-443a-b325-7db67281deb1", "date": "Tue, 16 Dec 2025 03:00:09 GMT"} })
14462025-12-16T03:00:09.956ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 71.594207ms iteration = 5 status = {}
14472025-12-16T03:00:09.956ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
14482025-12-16T03:00:09.956ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
14492025-12-16T03:00:09.976ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), time_created: 2025-12-16T02:59:15.195809Z, time_modified: 2025-12-16T02:59:15.195809Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
14502025-12-16T03:00:09.976ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
14512025-12-16T03:00:10.008ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14522025-12-16T03:00:10.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14532025-12-16T03:00:10.012ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14542025-12-16T03:00:10.012ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14552025-12-16T03:00:10.014ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14562025-12-16T03:00:10.017ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
14572025-12-16T03:00:10.017ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
14582025-12-16T03:00:10.021ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "b0327864-2b96-4c26-ae35-e69e9cda5429", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
14592025-12-16T03:00:10.023ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
14602025-12-16T03:00:10.023ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fddd842f-0dc2-4433-afc8-0d922c7d270d step component = Dns step id = Ensure
14612025-12-16T03:00:10.023ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
14622025-12-16T03:00:10.023ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14632025-12-16T03:00:10.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14642025-12-16T03:00:10.024ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14652025-12-16T03:00:10.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14662025-12-16T03:00:10.026ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14672025-12-16T03:00:10.029ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
14682025-12-16T03:00:10.030ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
14692025-12-16T03:00:10.030ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
14702025-12-16T03:00:10.030ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
14712025-12-16T03:00:10.030ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14722025-12-16T03:00:10.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14732025-12-16T03:00:10.032ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14742025-12-16T03:00:10.035ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
14752025-12-16T03:00:10.035ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14762025-12-16T03:00:10.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14772025-12-16T03:00:10.038ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
14782025-12-16T03:00:10.038ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
14792025-12-16T03:00:10.038ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14802025-12-16T03:00:10.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14812025-12-16T03:00:10.040ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14822025-12-16T03:00:10.043ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
14832025-12-16T03:00:10.043ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14842025-12-16T03:00:10.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14852025-12-16T03:00:10.045ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
14862025-12-16T03:00:10.045ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
14872025-12-16T03:00:10.045ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
14882025-12-16T03:00:10.045ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14892025-12-16T03:00:10.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14902025-12-16T03:00:10.046ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14912025-12-16T03:00:10.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14922025-12-16T03:00:10.048ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14932025-12-16T03:00:10.051ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
14942025-12-16T03:00:10.052ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
14952025-12-16T03:00:10.052ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
14962025-12-16T03:00:10.052ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
14972025-12-16T03:00:10.052ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14982025-12-16T03:00:10.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14992025-12-16T03:00:10.054ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15002025-12-16T03:00:10.059ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
15012025-12-16T03:00:10.059ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
15022025-12-16T03:00:10.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
15032025-12-16T03:00:10.062ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
15042025-12-16T03:00:10.062ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
15052025-12-16T03:00:10.062ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
15062025-12-16T03:00:10.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
15072025-12-16T03:00:10.063ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15082025-12-16T03:00:10.066ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
15092025-12-16T03:00:10.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
15102025-12-16T03:00:10.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
15112025-12-16T03:00:10.068ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
15122025-12-16T03:00:10.068ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
15132025-12-16T03:00:10.069ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
15142025-12-16T03:00:10.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
15152025-12-16T03:00:10.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15162025-12-16T03:00:10.073ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
15172025-12-16T03:00:10.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
15182025-12-16T03:00:10.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
15192025-12-16T03:00:10.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
15202025-12-16T03:00:10.076ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "755dfa4b-56f9-40eb-84be-18965929509f", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
15212025-12-16T03:00:10.076ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
15222025-12-16T03:00:10.076ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
15232025-12-16T03:00:10.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
15242025-12-16T03:00:10.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
15252025-12-16T03:00:10.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
15262025-12-16T03:00:10.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15272025-12-16T03:00:10.082ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
15282025-12-16T03:00:10.082ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "57a255e0-f736-4007-9ab6-1c180c037682", "content-length": "682", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
15292025-12-16T03:00:10.082ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15302025-12-16T03:00:10.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15312025-12-16T03:00:10.084ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
15322025-12-16T03:00:10.084ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
15332025-12-16T03:00:10.084ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fddd842f-0dc2-4433-afc8-0d922c7d270d step component = OmicronZones step id = Cleanup
15342025-12-16T03:00:10.084ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fddd842f-0dc2-4433-afc8-0d922c7d270d step component = OmicronZones step id = Cleanup
15352025-12-16T03:00:10.084ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fddd842f-0dc2-4433-afc8-0d922c7d270d step component = PhysicalDisks step id = Cleanup
15362025-12-16T03:00:10.084ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fddd842f-0dc2-4433-afc8-0d922c7d270d step component = Clickhouse step id = Ensure
15372025-12-16T03:00:10.084ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fddd842f-0dc2-4433-afc8-0d922c7d270d step component = Clickhouse step id = Ensure
15382025-12-16T03:00:10.101ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15392025-12-16T03:00:10.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15402025-12-16T03:00:10.104ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15412025-12-16T03:00:10.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15422025-12-16T03:00:10.107ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15432025-12-16T03:00:10.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15442025-12-16T03:00:10.115ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
15452025-12-16T03:00:10.116ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 503, headers: {"content-type": "application/json", "x-request-id": "416ee1f3-1727-41e5-98aa-e9b6756bf90b", "content-length": "94", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
15462025-12-16T03:00:10.118ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fddd842f-0dc2-4433-afc8-0d922c7d270d step component = SupportBundles step id = Cleanup
15472025-12-16T03:00:10.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
15482025-12-16T03:00:10.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
15492025-12-16T03:00:10.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15502025-12-16T03:00:10.124ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 915.456909ms iteration = 3 status = {}
15512025-12-16T03:00:10.124ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
15522025-12-16T03:00:10.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
15532025-12-16T03:00:10.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
15542025-12-16T03:00:10.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
15552025-12-16T03:00:10.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
15562025-12-16T03:00:10.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15572025-12-16T03:00:10.131ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
15582025-12-16T03:00:10.136ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fddd842f-0dc2-4433-afc8-0d922c7d270d step component = OmicronZones step id = Cleanup
15592025-12-16T03:00:10.136ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
15602025-12-16T03:00:10.136ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
15612025-12-16T03:00:10.136ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
15622025-12-16T03:00:10.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
15632025-12-16T03:00:10.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15642025-12-16T03:00:10.139ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
15652025-12-16T03:00:10.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
15662025-12-16T03:00:10.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
15672025-12-16T03:00:10.178ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
15682025-12-16T03:00:10.178ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(a91734db-9ac7-4793-9c32-ddbb21f8d986 (blueprint), false)
15692025-12-16T03:00:10.178ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fddd842f-0dc2-4433-afc8-0d922c7d270d step component = Cockroach step id = Ensure
15702025-12-16T03:00:10.178ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
15712025-12-16T03:00:10.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
15722025-12-16T03:00:10.204ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
15732025-12-16T03:00:10.204ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fddd842f-0dc2-4433-afc8-0d922c7d270d step component = MgsUpdates step id = Ensure
15742025-12-16T03:00:10.204ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 3 reason = Dependency
15752025-12-16T03:00:10.204ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 5 reason = Dependency
15762025-12-16T03:00:10.204ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
15772025-12-16T03:00:10.204ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15782025-12-16T03:00:10.204ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.037016699s iteration = 3 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"3b7b2c09-a827-4c7f-9649-b21b8c114695","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"3b7b2c09-a827-4c7f-9649-b21b8c114695","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":4430,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":82997492,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":82997452,"secs":0}},"event_index":1,"execution_id":"3b7b2c09-a827-4c7f-9649-b21b8c114695","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":83018352,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":235883778,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":235883738,"secs":0}},"event_index":2,"execution_id":"3b7b2c09-a827-4c7f-9649-b21b8c114695","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":318921190,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":66722526,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":66722486,"secs":0}},"event_index":3,"execution_id":"3b7b2c09-a827-4c7f-9649-b21b8c114695","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":385659796,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":218932214,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":218932174,"secs":0}},"event_index":4,"execution_id":"3b7b2c09-a827-4c7f-9649-b21b8c114695","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":604606840,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":251547348,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":251547318,"secs":0}},"event_index":5,"execution_id":"3b7b2c09-a827-4c7f-9649-b21b8c114695","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":856168708,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":60456306,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":60456276,"secs":0}},"event_index":6,"execution_id":"3b7b2c09-a827-4c7f-9649-b21b8c114695","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":916639514,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4930,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4890,"secs":0}},"event_index":7,"execution_id":"3b7b2c09-a827-4c7f-9649-b21b8c114695","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":916658384,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2790,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2760,"secs":0}},"event_index":8,"execution_id":"3b7b2c09-a827-4c7f-9649-b21b8c114695","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":916663704,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3190,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3160,"secs":0}},"event_index":9,"execution_id":"3b7b2c09-a827-4c7f-9649-b21b8c114695","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":916669184,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1750,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1710,"secs":0}},"event_index":10,"execution_id":"3b7b2c09-a827-4c7f-9649-b21b8c114695","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":916672614,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":34310635,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"warning","message":"failed to initialize single-node clickhouse database: Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"416ee1f3-1727-41e5-98aa-e9b6756bf90b\\", \\"content-length\\": \\"94\\", \\"date\\": \\"Tue, 16 Dec 2025 03:00:10 GMT\\"}; value: Error { error_code: None, message: \\"Service Unavailable\\", request_id: \\"416ee1f3-1727-41e5-98aa-e9b6756bf90b\\" }","metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":34310605,"secs":0}},"event_index":11,"execution_id":"3b7b2c09-a827-4c7f-9649-b21b8c114695","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":950988949,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":17721305,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":17721275,"secs":0}},"event_index":12,"execution_id":"3b7b2c09-a827-4c7f-9649-b21b8c114695","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":968724414,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":42587719,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":42587679,"secs":0}},"event_index":13,"execution_id":"3b7b2c09-a827-4c7f-9649-b21b8c114695","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":11325333,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":25323031,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":25323000,"secs":0}},"event_index":14,"execution_id":"3b7b2c09-a827-4c7f-9649-b21b8c114695","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":36662765,"secs":1}},{"data":{"attempt_elapsed":{"nanos":4750,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4720,"secs":0}},"event_index":15,"execution_id":"3b7b2c09-a827-4c7f-9649-b21b8c114695","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":36676595,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986"}
15792025-12-16T03:00:10.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
15802025-12-16T03:00:10.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
15812025-12-16T03:00:10.206ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15822025-12-16T03:00:10.206ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15832025-12-16T03:00:10.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
15842025-12-16T03:00:10.209ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15852025-12-16T03:00:10.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
15862025-12-16T03:00:10.209ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
15872025-12-16T03:00:10.209ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
15882025-12-16T03:00:10.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
15892025-12-16T03:00:10.211ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15902025-12-16T03:00:10.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
15912025-12-16T03:00:10.212ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15922025-12-16T03:00:10.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
15932025-12-16T03:00:10.214ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15942025-12-16T03:00:10.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
15952025-12-16T03:00:10.216ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
15962025-12-16T03:00:10.216ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 12.182275ms iteration = 5 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
15972025-12-16T03:00:10.230ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(f95f7584-07a2-4e32-99ff-a99540f32823, 0), (2244486f-2a09-4cf4-9f19-e880b9c3540b, 0), (a695ac0f-35a9-43ea-9255-94bc1405678c, 0), (2ff2c680-3e77-43d1-87e0-51edcb3fda30, 0)] collection_id = f95f7584-07a2-4e32-99ff-a99540f32823
15982025-12-16T03:00:10.231ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
15992025-12-16T03:00:10.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
16002025-12-16T03:00:10.352ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = f95f7584-07a2-4e32-99ff-a99540f32823 file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
16012025-12-16T03:00:10.352ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16022025-12-16T03:00:10.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
16032025-12-16T03:00:10.355ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(2244486f-2a09-4cf4-9f19-e880b9c3540b, 0), (a695ac0f-35a9-43ea-9255-94bc1405678c, 0), (2ff2c680-3e77-43d1-87e0-51edcb3fda30, 0)]
16042025-12-16T03:00:10.356ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1017, tv_nsec: 974301775 } })
16052025-12-16T03:00:10.388ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("20488683-5778-4090-b36e-e6f0d80a682b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("35dbaad1-8536-44a0-8a0a-35d7c39dcfbb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b57fa226-2428-4a54-b85d-60409e28b484.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("d812a3e2-48ae-4812-a1cc-7eb7623e4e00.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::5)) }], valid_until: Instant { tv_sec: 1018, tv_nsec: 5821474 } })
16062025-12-16T03:00:10.388ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
16072025-12-16T03:00:10.388ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
16082025-12-16T03:00:10.388ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
16092025-12-16T03:00:10.389ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "974885dc-9bd7-4ca8-b0cf-22a896ccfbcc", "content-length": "476", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16102025-12-16T03:00:10.389ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
16112025-12-16T03:00:10.389ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d54ed17f-155b-4896-8ff8-5ad0e5fb1d8c", "content-length": "734", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16122025-12-16T03:00:10.389ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
16132025-12-16T03:00:10.390ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e4fa0ad-120f-4868-ab33-30de49f0b4cf", "content-length": "111", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16142025-12-16T03:00:10.390ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
16152025-12-16T03:00:10.390ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "72457d23-0f4d-40e4-9068-fb8d9fa1cfcf", "content-length": "111", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16162025-12-16T03:00:10.390ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
16172025-12-16T03:00:10.390ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "febb00c4-395d-4df1-a7e6-c0d9001f89be", "content-length": "163", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16182025-12-16T03:00:10.390ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
16192025-12-16T03:00:10.391ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9f4e2c0-e20f-49e5-b9a1-1f126bca53f5", "content-length": "163", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16202025-12-16T03:00:10.391ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
16212025-12-16T03:00:10.391ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6b3c8d7-72b5-4f73-9ff5-6d01aebe7907", "content-length": "166", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16222025-12-16T03:00:10.391ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
16232025-12-16T03:00:10.392ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "79260cbe-4c2e-423b-9683-a4552daf112a", "content-length": "166", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16242025-12-16T03:00:10.392ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
16252025-12-16T03:00:10.392ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ebc3120-418f-4b61-8335-940af814f71d", "content-length": "702", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16262025-12-16T03:00:10.392ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
16272025-12-16T03:00:10.392ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a581d9b-6210-4fae-92ab-5646594e48dd", "content-length": "727", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16282025-12-16T03:00:10.392ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
16292025-12-16T03:00:10.392ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f7ad061-909e-4963-aac4-6298ec8c7419", "content-length": "729", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16302025-12-16T03:00:10.392ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
16312025-12-16T03:00:10.392ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b212864-7eae-4985-b9ca-6e280c30f917", "content-length": "728", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16322025-12-16T03:00:10.392ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
16332025-12-16T03:00:10.393ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "10db682f-c455-439c-a1a1-162ed7ec8a30", "content-length": "722", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16342025-12-16T03:00:10.393ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
16352025-12-16T03:00:10.393ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "736b2bd3-6816-4165-badd-ff925dd5c394", "content-length": "10", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16362025-12-16T03:00:10.393ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
16372025-12-16T03:00:10.393ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "52614cfc-a05d-4168-8f0d-7fdf2412deb5", "content-length": "144", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16382025-12-16T03:00:10.393ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
16392025-12-16T03:00:10.393ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e867fa90-6670-4f01-a3bd-a35bb37b7753", "content-length": "147", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16402025-12-16T03:00:10.393ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
16412025-12-16T03:00:10.393ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0e6be1c-240a-4bba-bd4e-5b8c07b5baf1", "content-length": "109", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16422025-12-16T03:00:10.394ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
16432025-12-16T03:00:10.394ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb4d27f7-53ed-446a-8290-34db8812a993", "content-length": "109", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16442025-12-16T03:00:10.394ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
16452025-12-16T03:00:10.394ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "52cfd78c-2768-4634-9a87-24a8b1164da6", "content-length": "163", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16462025-12-16T03:00:10.394ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
16472025-12-16T03:00:10.395ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2063c3fe-e396-4d68-9a0b-2e4644155aa7", "content-length": "163", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16482025-12-16T03:00:10.395ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
16492025-12-16T03:00:10.395ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5314f005-0f30-45f6-86dc-29589a27b1f7", "content-length": "166", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16502025-12-16T03:00:10.395ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
16512025-12-16T03:00:10.395ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ddf22838-ae72-414b-979f-9720db57ef7a", "content-length": "166", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16522025-12-16T03:00:10.395ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
16532025-12-16T03:00:10.395ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8e396c9-6d8f-4dfe-b60b-d10c931567b8", "content-length": "702", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16542025-12-16T03:00:10.395ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
16552025-12-16T03:00:10.396ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8cc513cb-d9b9-47e4-9610-a9a91cba407f", "content-length": "727", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16562025-12-16T03:00:10.396ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
16572025-12-16T03:00:10.396ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2e18932-2d02-4fa6-a743-c83653db987e", "content-length": "729", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16582025-12-16T03:00:10.396ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
16592025-12-16T03:00:10.396ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b563151-165f-4202-afc3-322df6ba4fbb", "content-length": "728", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16602025-12-16T03:00:10.396ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16612025-12-16T03:00:10.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
16622025-12-16T03:00:10.398ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16632025-12-16T03:00:10.400ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
16642025-12-16T03:00:10.400ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16652025-12-16T03:00:10.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
16662025-12-16T03:00:10.402ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16672025-12-16T03:00:10.404ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
16682025-12-16T03:00:10.404ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16692025-12-16T03:00:10.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
16702025-12-16T03:00:10.437ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
16712025-12-16T03:00:10.437ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
16722025-12-16T03:00:10.489ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "13f22060-2785-42d8-8802-2461dae8f65b", "content-length": "95609", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16732025-12-16T03:00:10.491ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
16742025-12-16T03:00:10.491ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
16752025-12-16T03:00:10.491ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
16762025-12-16T03:00:10.491ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
16772025-12-16T03:00:10.491ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
16782025-12-16T03:00:10.491ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
16792025-12-16T03:00:10.491ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
16802025-12-16T03:00:10.491ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
16812025-12-16T03:00:10.492ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fba984c5-29fa-4598-bd9a-005a89bb065e", "content-length": "64", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16822025-12-16T03:00:10.492ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
16832025-12-16T03:00:10.492ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9df92cee-3ddc-4ec0-8619-34cafea19ceb", "content-length": "64", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16842025-12-16T03:00:10.492ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a1e08d4-908d-43ba-b4a9-faca35da28dd", "content-length": "64", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16852025-12-16T03:00:10.492ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
16862025-12-16T03:00:10.492ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b28da3c-9bb8-42af-aea8-12a9a2057506", "content-length": "64", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16872025-12-16T03:00:10.492ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
16882025-12-16T03:00:10.492ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
16892025-12-16T03:00:10.492ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "70b1ae94-488d-4a46-9bf1-c971f9fbe3b2", "content-length": "64", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16902025-12-16T03:00:10.492ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
16912025-12-16T03:00:10.532ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1648b70-5849-4b0e-9798-80df998f9308", "content-length": "381191", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16922025-12-16T03:00:10.535ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3b036fe-e289-4490-bf12-809b6fde48a8", "content-length": "374148", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16932025-12-16T03:00:10.536ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "de458663-2f8e-4306-b5d0-d37c90ff1f16", "content-length": "414002", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16942025-12-16T03:00:10.537ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "75888246-4c28-4a18-af49-dccc3db5fcdf", "content-length": "376053", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16952025-12-16T03:00:10.538ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "63f41da8-34d7-4573-bb07-212c01070a99", "content-length": "377736", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
16962025-12-16T03:00:10.542ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
16972025-12-16T03:00:10.542ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
16982025-12-16T03:00:10.542ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
16992025-12-16T03:00:10.542ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
17002025-12-16T03:00:10.542ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
17012025-12-16T03:00:10.577ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c (omicron_zone)
17022025-12-16T03:00:10.577ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
17032025-12-16T03:00:10.581ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "a01b8266-ce50-473d-a112-a7c2fe427b2c", "content-length": "124", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
17042025-12-16T03:00:10.583ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
17052025-12-16T03:00:10.675ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
17062025-12-16T03:00:10.675ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
17072025-12-16T03:00:10.676ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "21d66136-db96-4521-b55a-d4f55b8c3e0c", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
17082025-12-16T03:00:10.676ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
17092025-12-16T03:00:10.677ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
17102025-12-16T03:00:10.677ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
17112025-12-16T03:00:10.677ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d849af0-d68d-4c0b-9e9e-c91fefdea1fe", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
17122025-12-16T03:00:10.677ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
17132025-12-16T03:00:10.679ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
17142025-12-16T03:00:10.679ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
17152025-12-16T03:00:10.679ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f82818f-c2dd-4ed1-ac55-2d063bd0952d", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:00:10 GMT"} })
17162025-12-16T03:00:10.679ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
17172025-12-16T03:00:10.681ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
17182025-12-16T03:00:10.681ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
17192025-12-16T03:00:10.683ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
17202025-12-16T03:00:10.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
17212025-12-16T03:00:10.685ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17222025-12-16T03:00:10.687ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
17232025-12-16T03:00:10.687ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
17242025-12-16T03:00:10.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
17252025-12-16T03:00:10.894ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 1e5906a3-cfae-4815-8c76-8082f2ae9f87 file = nexus/db-queries/src/db/datastore/inventory.rs:1613
17262025-12-16T03:00:10.894ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 1e5906a3-cfae-4815-8c76-8082f2ae9f87 time_started = 2025-12-16 03:00:10.388724 UTC
17272025-12-16T03:00:10.894ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 7 reason = Dependency
17282025-12-16T03:00:10.894ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
17292025-12-16T03:00:10.894ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 8 reason = Dependency
17302025-12-16T03:00:10.894ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 15.42\u{b5}s iteration = 7 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
17312025-12-16T03:00:10.894ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
17322025-12-16T03:00:10.894ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 690.072243ms iteration = 3 status = {"collection_id":"1e5906a3-cfae-4815-8c76-8082f2ae9f87","time_done":"2025-12-16 03:00:10.681388 UTC","time_started":"2025-12-16 03:00:10.388724 UTC"}
17332025-12-16T03:00:10.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
17342025-12-16T03:00:10.896ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17352025-12-16T03:00:10.900ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
17362025-12-16T03:00:10.900ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
17372025-12-16T03:00:10.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
17382025-12-16T03:00:11.157ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
17392025-12-16T03:00:11.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
17402025-12-16T03:00:11.290ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 396.063478ms iteration = 8 status = {"Loaded":{"collection_id":"663629f5-b2a5-4998-9d76-28f70bfaf87e","time_loaded":"2025-12-16T03:00:10.894379997Z","time_started":"2025-12-16T03:00:10.446822Z"}}
17412025-12-16T03:00:11.290ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 8 reason = Dependency
17422025-12-16T03:00:11.290ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 6 reason = Dependency
17432025-12-16T03:00:11.290ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
17442025-12-16T03:00:11.290ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 7 reason = Dependency
17452025-12-16T03:00:11.290ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 23.62\u{b5}s iteration = 8 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
17462025-12-16T03:00:11.290ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 4 reason = Dependency
17472025-12-16T03:00:11.290ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17482025-12-16T03:00:11.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17492025-12-16T03:00:11.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17502025-12-16T03:00:11.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17512025-12-16T03:00:11.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17522025-12-16T03:00:11.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17532025-12-16T03:00:11.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17542025-12-16T03:00:11.293ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17552025-12-16T03:00:11.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17562025-12-16T03:00:11.296ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
17572025-12-16T03:00:11.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17582025-12-16T03:00:11.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17592025-12-16T03:00:11.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
17602025-12-16T03:00:11.296ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17612025-12-16T03:00:11.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17622025-12-16T03:00:11.296ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
17632025-12-16T03:00:11.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17642025-12-16T03:00:11.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17652025-12-16T03:00:11.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17662025-12-16T03:00:11.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17672025-12-16T03:00:11.298ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17682025-12-16T03:00:11.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17692025-12-16T03:00:11.299ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
17702025-12-16T03:00:11.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17712025-12-16T03:00:11.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17722025-12-16T03:00:11.301ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
17732025-12-16T03:00:11.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17742025-12-16T03:00:11.301ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17752025-12-16T03:00:11.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17762025-12-16T03:00:11.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17772025-12-16T03:00:11.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17782025-12-16T03:00:11.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17792025-12-16T03:00:11.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17802025-12-16T03:00:11.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
17812025-12-16T03:00:11.304ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17822025-12-16T03:00:11.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17832025-12-16T03:00:11.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17842025-12-16T03:00:11.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17852025-12-16T03:00:11.308ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
17862025-12-16T03:00:11.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17872025-12-16T03:00:11.308ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
17882025-12-16T03:00:11.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17892025-12-16T03:00:11.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17902025-12-16T03:00:11.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17912025-12-16T03:00:11.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17922025-12-16T03:00:11.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17932025-12-16T03:00:11.312ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
17942025-12-16T03:00:11.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17952025-12-16T03:00:11.312ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
17962025-12-16T03:00:11.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17972025-12-16T03:00:11.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17982025-12-16T03:00:11.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17992025-12-16T03:00:11.313ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 22.997801ms iteration = 6 status = {"physical_disks_added":0}
18002025-12-16T03:00:11.315ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
18012025-12-16T03:00:11.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
18022025-12-16T03:00:11.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
18032025-12-16T03:00:11.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18042025-12-16T03:00:11.318ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
18052025-12-16T03:00:11.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
18062025-12-16T03:00:11.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
18072025-12-16T03:00:11.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18082025-12-16T03:00:11.322ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
18092025-12-16T03:00:11.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
18102025-12-16T03:00:11.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
18112025-12-16T03:00:11.324ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
18122025-12-16T03:00:11.324ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 34.114212ms iteration = 4 status = {"blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","inventory_collection_id":"663629f5-b2a5-4998-9d76-28f70bfaf87e","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
18132025-12-16T03:00:11.351ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18142025-12-16T03:00:11.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18152025-12-16T03:00:11.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18162025-12-16T03:00:11.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18172025-12-16T03:00:11.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18182025-12-16T03:00:11.358ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
18192025-12-16T03:00:11.358ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18202025-12-16T03:00:11.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18212025-12-16T03:00:11.360ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18222025-12-16T03:00:11.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18232025-12-16T03:00:11.362ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18242025-12-16T03:00:11.365ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
18252025-12-16T03:00:11.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18262025-12-16T03:00:11.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18272025-12-16T03:00:11.367ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18282025-12-16T03:00:11.370ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
18292025-12-16T03:00:11.370ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18302025-12-16T03:00:11.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18312025-12-16T03:00:11.372ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18322025-12-16T03:00:11.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18332025-12-16T03:00:11.373ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18342025-12-16T03:00:11.376ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
18352025-12-16T03:00:11.376ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18362025-12-16T03:00:11.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18372025-12-16T03:00:11.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18382025-12-16T03:00:11.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18392025-12-16T03:00:11.380ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18402025-12-16T03:00:11.383ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
18412025-12-16T03:00:11.383ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18422025-12-16T03:00:11.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18432025-12-16T03:00:11.385ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18442025-12-16T03:00:11.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18452025-12-16T03:00:11.387ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18462025-12-16T03:00:11.390ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
18472025-12-16T03:00:11.390ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18482025-12-16T03:00:11.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18492025-12-16T03:00:11.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18502025-12-16T03:00:11.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18512025-12-16T03:00:11.404ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18522025-12-16T03:00:11.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18532025-12-16T03:00:11.406ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18542025-12-16T03:00:11.409ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
18552025-12-16T03:00:11.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18562025-12-16T03:00:11.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18572025-12-16T03:00:11.410ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18582025-12-16T03:00:11.413ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
18592025-12-16T03:00:11.413ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18602025-12-16T03:00:11.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18612025-12-16T03:00:11.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18622025-12-16T03:00:11.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18632025-12-16T03:00:11.417ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18642025-12-16T03:00:11.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18652025-12-16T03:00:11.418ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18662025-12-16T03:00:11.421ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
18672025-12-16T03:00:11.421ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18682025-12-16T03:00:11.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18692025-12-16T03:00:11.423ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18702025-12-16T03:00:11.426ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
18712025-12-16T03:00:11.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18722025-12-16T03:00:11.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18732025-12-16T03:00:11.428ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18742025-12-16T03:00:11.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18752025-12-16T03:00:11.429ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18762025-12-16T03:00:11.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18772025-12-16T03:00:11.430ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18782025-12-16T03:00:11.434ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
18792025-12-16T03:00:11.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18802025-12-16T03:00:11.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18812025-12-16T03:00:11.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18822025-12-16T03:00:11.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18832025-12-16T03:00:11.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18842025-12-16T03:00:11.440ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
18852025-12-16T03:00:11.441ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18862025-12-16T03:00:11.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18872025-12-16T03:00:11.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18882025-12-16T03:00:11.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18892025-12-16T03:00:11.445ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18902025-12-16T03:00:11.447ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
18912025-12-16T03:00:11.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18922025-12-16T03:00:11.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18932025-12-16T03:00:11.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18942025-12-16T03:00:11.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18952025-12-16T03:00:11.450ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18962025-12-16T03:00:11.453ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
18972025-12-16T03:00:11.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18982025-12-16T03:00:11.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18992025-12-16T03:00:11.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19002025-12-16T03:00:11.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19012025-12-16T03:00:11.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19022025-12-16T03:00:11.459ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
19032025-12-16T03:00:11.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19042025-12-16T03:00:11.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19052025-12-16T03:00:11.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19062025-12-16T03:00:11.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19072025-12-16T03:00:11.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19082025-12-16T03:00:11.464ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
19092025-12-16T03:00:11.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19102025-12-16T03:00:11.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19112025-12-16T03:00:11.466ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
19122025-12-16T03:00:11.466ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
19132025-12-16T03:00:11.466ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = f6702ec3-a715-4a8a-8bba-8930b2b62a5b parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
19142025-12-16T03:00:11.466ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19152025-12-16T03:00:11.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19162025-12-16T03:00:11.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19172025-12-16T03:00:11.470ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
19182025-12-16T03:00:11.470ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
19192025-12-16T03:00:11.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
19202025-12-16T03:00:11.473ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
19212025-12-16T03:00:11.473ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
19222025-12-16T03:00:11.473ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 183.034118ms iteration = 7 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
19232025-12-16T03:00:14.208ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 14 reason = Timeout
19242025-12-16T03:00:14.208ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
19252025-12-16T03:00:14.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
19262025-12-16T03:00:14.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19272025-12-16T03:00:14.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
19282025-12-16T03:00:14.214ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
19292025-12-16T03:00:14.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
19302025-12-16T03:00:14.215ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
19312025-12-16T03:00:14.215ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.249482ms iteration = 14 status = {"config_updated":false}
19322025-12-16T03:00:19.208ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 8 reason = Timeout
19332025-12-16T03:00:19.208ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 15 reason = Timeout
19342025-12-16T03:00:19.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19352025-12-16T03:00:19.208ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
19362025-12-16T03:00:19.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
19372025-12-16T03:00:19.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
19382025-12-16T03:00:19.208ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 8 reason = Timeout
19392025-12-16T03:00:19.208ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19402025-12-16T03:00:19.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19412025-12-16T03:00:19.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19422025-12-16T03:00:19.210ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19432025-12-16T03:00:19.210ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19442025-12-16T03:00:19.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19452025-12-16T03:00:19.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19462025-12-16T03:00:19.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19472025-12-16T03:00:19.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
19482025-12-16T03:00:19.214ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
19492025-12-16T03:00:19.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
19502025-12-16T03:00:19.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
19512025-12-16T03:00:19.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19522025-12-16T03:00:19.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
19532025-12-16T03:00:19.216ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
19542025-12-16T03:00:19.216ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.372856ms iteration = 15 status = {"config_updated":false}
19552025-12-16T03:00:19.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19562025-12-16T03:00:19.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
19572025-12-16T03:00:19.221ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19582025-12-16T03:00:19.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
19592025-12-16T03:00:19.234ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
19602025-12-16T03:00:19.234ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.03777ms iteration = 8 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
19612025-12-16T03:00:19.254ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 45.75102ms iteration = 8 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
19622025-12-16T03:00:21.120ZINFOnexus (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:54115 remote_addr = [fd00:1122:3344:101::d]:43062
19632025-12-16T03:00:21.128ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 8052 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 77676825-3519-4b5e-8712-56fa73e6d812 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
19642025-12-16T03:00:24.208ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 9 reason = Timeout
19652025-12-16T03:00:24.208ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 16 reason = Timeout
19662025-12-16T03:00:24.208ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
19672025-12-16T03:00:24.208ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
19682025-12-16T03:00:24.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
19692025-12-16T03:00:24.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
19702025-12-16T03:00:24.211ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19712025-12-16T03:00:24.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19722025-12-16T03:00:24.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
19732025-12-16T03:00:24.215ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
19742025-12-16T03:00:24.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
19752025-12-16T03:00:24.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
19762025-12-16T03:00:24.215ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
19772025-12-16T03:00:24.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
19782025-12-16T03:00:24.217ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
19792025-12-16T03:00:24.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
19802025-12-16T03:00:24.217ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
19812025-12-16T03:00:24.217ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 9.075275ms iteration = 16 status = {"config_updated":false}
19822025-12-16T03:00:24.226ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 6 reason = Timeout
19832025-12-16T03:00:24.226ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 6 reason = Timeout
19842025-12-16T03:00:24.226ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
19852025-12-16T03:00:24.226ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
19862025-12-16T03:00:24.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
19872025-12-16T03:00:24.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
19882025-12-16T03:00:24.226ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
19892025-12-16T03:00:24.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
19902025-12-16T03:00:24.227ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19912025-12-16T03:00:24.228ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19922025-12-16T03:00:24.229ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
19932025-12-16T03:00:24.229ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
19942025-12-16T03:00:24.230ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
19952025-12-16T03:00:24.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
19962025-12-16T03:00:24.231ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
19972025-12-16T03:00:24.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
19982025-12-16T03:00:24.232ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.995516ms iteration = 6 status = "NoSitrep"
19992025-12-16T03:00:24.232ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
20002025-12-16T03:00:24.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
20012025-12-16T03:00:24.233ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20022025-12-16T03:00:24.235ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
20032025-12-16T03:00:24.235ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
20042025-12-16T03:00:24.235ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 9.795355ms iteration = 6 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
20052025-12-16T03:00:24.254ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 5 reason = Dependency
20062025-12-16T03:00:24.254ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 45.519837ms iteration = 9 status = {"Loaded":{"collection_id":"993293d3-f4d1-4544-a5c6-b0c29460233f","time_loaded":"2025-12-16T03:00:24.208591607Z","time_started":"2025-12-16T03:00:10.647828Z"}}
20072025-12-16T03:00:24.254ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 9 reason = Dependency
20082025-12-16T03:00:24.254ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 8 reason = Dependency
20092025-12-16T03:00:24.254ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
20102025-12-16T03:00:24.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
20112025-12-16T03:00:24.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20122025-12-16T03:00:24.254ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 10.89\u{b5}s iteration = 9 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
20132025-12-16T03:00:24.254ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 7 reason = Dependency
20142025-12-16T03:00:24.254ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20152025-12-16T03:00:24.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
20162025-12-16T03:00:24.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20172025-12-16T03:00:24.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20182025-12-16T03:00:24.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20192025-12-16T03:00:24.255ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20202025-12-16T03:00:24.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20212025-12-16T03:00:24.259ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
20222025-12-16T03:00:24.259ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20232025-12-16T03:00:24.259ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
20242025-12-16T03:00:24.259ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20252025-12-16T03:00:24.259ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
20262025-12-16T03:00:24.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20272025-12-16T03:00:24.259ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
20282025-12-16T03:00:24.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20292025-12-16T03:00:24.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
20302025-12-16T03:00:24.261ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20312025-12-16T03:00:24.261ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20322025-12-16T03:00:24.261ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20332025-12-16T03:00:24.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20342025-12-16T03:00:24.262ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20352025-12-16T03:00:24.264ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
20362025-12-16T03:00:24.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
20372025-12-16T03:00:24.264ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
20382025-12-16T03:00:24.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20392025-12-16T03:00:24.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
20402025-12-16T03:00:24.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20412025-12-16T03:00:24.265ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
20422025-12-16T03:00:24.265ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20432025-12-16T03:00:24.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20442025-12-16T03:00:24.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
20452025-12-16T03:00:24.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
20462025-12-16T03:00:24.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20472025-12-16T03:00:24.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20482025-12-16T03:00:24.268ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20492025-12-16T03:00:24.268ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20502025-12-16T03:00:24.270ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
20512025-12-16T03:00:24.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
20522025-12-16T03:00:24.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
20532025-12-16T03:00:24.271ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
20542025-12-16T03:00:24.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20552025-12-16T03:00:24.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20562025-12-16T03:00:24.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20572025-12-16T03:00:24.272ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20582025-12-16T03:00:24.273ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 19.581537ms iteration = 7 status = {"physical_disks_added":0}
20592025-12-16T03:00:24.274ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
20602025-12-16T03:00:24.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
20612025-12-16T03:00:24.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
20622025-12-16T03:00:24.274ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
20632025-12-16T03:00:24.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20642025-12-16T03:00:24.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20652025-12-16T03:00:24.276ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
20662025-12-16T03:00:24.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
20672025-12-16T03:00:24.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
20682025-12-16T03:00:24.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20692025-12-16T03:00:24.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20702025-12-16T03:00:24.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20712025-12-16T03:00:24.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20722025-12-16T03:00:24.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20732025-12-16T03:00:24.280ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
20742025-12-16T03:00:24.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
20752025-12-16T03:00:24.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
20762025-12-16T03:00:24.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20772025-12-16T03:00:24.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20782025-12-16T03:00:24.284ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
20792025-12-16T03:00:24.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
20802025-12-16T03:00:24.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
20812025-12-16T03:00:24.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
20822025-12-16T03:00:24.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20832025-12-16T03:00:24.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20842025-12-16T03:00:24.285ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
20852025-12-16T03:00:24.285ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 31.784163ms iteration = 5 status = {"blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","inventory_collection_id":"993293d3-f4d1-4544-a5c6-b0c29460233f","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
20862025-12-16T03:00:24.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20872025-12-16T03:00:24.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20882025-12-16T03:00:24.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20892025-12-16T03:00:24.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
20902025-12-16T03:00:24.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20912025-12-16T03:00:24.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20922025-12-16T03:00:24.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20932025-12-16T03:00:24.295ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
20942025-12-16T03:00:24.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20952025-12-16T03:00:24.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20962025-12-16T03:00:24.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
20972025-12-16T03:00:24.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
20982025-12-16T03:00:24.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20992025-12-16T03:00:24.301ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
21002025-12-16T03:00:24.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21012025-12-16T03:00:24.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21022025-12-16T03:00:24.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21032025-12-16T03:00:24.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21042025-12-16T03:00:24.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21052025-12-16T03:00:24.307ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
21062025-12-16T03:00:24.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21072025-12-16T03:00:24.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21082025-12-16T03:00:24.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21092025-12-16T03:00:24.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21102025-12-16T03:00:24.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21112025-12-16T03:00:24.313ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
21122025-12-16T03:00:24.313ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21132025-12-16T03:00:24.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21142025-12-16T03:00:24.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21152025-12-16T03:00:24.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21162025-12-16T03:00:24.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21172025-12-16T03:00:24.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21182025-12-16T03:00:24.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21192025-12-16T03:00:24.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
21202025-12-16T03:00:24.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21212025-12-16T03:00:24.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21222025-12-16T03:00:24.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21232025-12-16T03:00:24.333ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
21242025-12-16T03:00:24.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21252025-12-16T03:00:24.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21262025-12-16T03:00:24.335ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21272025-12-16T03:00:24.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21282025-12-16T03:00:24.336ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21292025-12-16T03:00:24.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21302025-12-16T03:00:24.338ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21312025-12-16T03:00:24.340ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
21322025-12-16T03:00:24.340ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21332025-12-16T03:00:24.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21342025-12-16T03:00:24.342ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21352025-12-16T03:00:24.345ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
21362025-12-16T03:00:24.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21372025-12-16T03:00:24.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21382025-12-16T03:00:24.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21392025-12-16T03:00:24.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21402025-12-16T03:00:24.348ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21412025-12-16T03:00:24.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21422025-12-16T03:00:24.349ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21432025-12-16T03:00:24.352ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
21442025-12-16T03:00:24.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21452025-12-16T03:00:24.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21462025-12-16T03:00:24.354ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21472025-12-16T03:00:24.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21482025-12-16T03:00:24.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21492025-12-16T03:00:24.358ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
21502025-12-16T03:00:24.359ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21512025-12-16T03:00:24.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21522025-12-16T03:00:24.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21532025-12-16T03:00:24.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21542025-12-16T03:00:24.362ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21552025-12-16T03:00:24.365ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
21562025-12-16T03:00:24.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21572025-12-16T03:00:24.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21582025-12-16T03:00:24.367ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21592025-12-16T03:00:24.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21602025-12-16T03:00:24.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21612025-12-16T03:00:24.371ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
21622025-12-16T03:00:24.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21632025-12-16T03:00:24.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21642025-12-16T03:00:24.373ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21652025-12-16T03:00:24.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21662025-12-16T03:00:24.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21672025-12-16T03:00:24.377ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
21682025-12-16T03:00:24.377ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21692025-12-16T03:00:24.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21702025-12-16T03:00:24.378ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21712025-12-16T03:00:24.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21722025-12-16T03:00:24.380ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21732025-12-16T03:00:24.382ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
21742025-12-16T03:00:24.382ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21752025-12-16T03:00:24.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21762025-12-16T03:00:24.384ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
21772025-12-16T03:00:24.384ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
21782025-12-16T03:00:24.384ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 5d5f4b83-2b61-45fe-8967-840b40949f62 parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
21792025-12-16T03:00:24.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21802025-12-16T03:00:24.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21812025-12-16T03:00:24.385ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21822025-12-16T03:00:24.388ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
21832025-12-16T03:00:24.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
21842025-12-16T03:00:24.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
21852025-12-16T03:00:24.390ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
21862025-12-16T03:00:24.390ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
21872025-12-16T03:00:24.391ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 136.886685ms iteration = 8 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
21882025-12-16T03:00:29.208ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 9 reason = Timeout
21892025-12-16T03:00:29.208ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 17 reason = Timeout
21902025-12-16T03:00:29.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21912025-12-16T03:00:29.208ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
21922025-12-16T03:00:29.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
21932025-12-16T03:00:29.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
21942025-12-16T03:00:29.209ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 9 reason = Timeout
21952025-12-16T03:00:29.209ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21962025-12-16T03:00:29.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21972025-12-16T03:00:29.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21982025-12-16T03:00:29.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21992025-12-16T03:00:29.211ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22002025-12-16T03:00:29.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22012025-12-16T03:00:29.212ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.380513ms iteration = 9 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
22022025-12-16T03:00:29.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
22032025-12-16T03:00:29.215ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
22042025-12-16T03:00:29.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
22052025-12-16T03:00:29.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22062025-12-16T03:00:29.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
22072025-12-16T03:00:29.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
22082025-12-16T03:00:29.217ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
22092025-12-16T03:00:29.217ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.741171ms iteration = 17 status = {"config_updated":false}
22102025-12-16T03:00:29.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22112025-12-16T03:00:29.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
22122025-12-16T03:00:29.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22132025-12-16T03:00:29.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
22142025-12-16T03:00:29.221ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22152025-12-16T03:00:29.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
22162025-12-16T03:00:29.234ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
22172025-12-16T03:00:29.234ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.714636ms iteration = 9 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
22182025-12-16T03:00:31.060ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 504 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 1bfde473-d85e-4049-9e88-b3c3b6c6a98d response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
22192025-12-16T03:00:34.209ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 18 reason = Timeout
22202025-12-16T03:00:34.209ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
22212025-12-16T03:00:34.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
22222025-12-16T03:00:34.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22232025-12-16T03:00:34.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
22242025-12-16T03:00:34.215ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
22252025-12-16T03:00:34.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
22262025-12-16T03:00:34.217ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
22272025-12-16T03:00:34.217ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.534938ms iteration = 18 status = {"config_updated":false}
22282025-12-16T03:00:39.167ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 4 reason = Timeout
22292025-12-16T03:00:39.167ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 4 reason = Timeout
22302025-12-16T03:00:39.167ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 4 reason = Timeout
22312025-12-16T03:00:39.167ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
22322025-12-16T03:00:39.167ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
22332025-12-16T03:00:39.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
22342025-12-16T03:00:39.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
22352025-12-16T03:00:39.172ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1046, tv_nsec: 789985539 } })
22362025-12-16T03:00:39.173ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
22372025-12-16T03:00:39.173ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1046, tv_nsec: 790999222 } })
22382025-12-16T03:00:39.180ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1046, tv_nsec: 797481304 } })
22392025-12-16T03:00:39.183ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 16.062344ms iteration = 4 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
22402025-12-16T03:00:39.208ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 10 reason = Timeout
22412025-12-16T03:00:39.208ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 10 reason = Timeout
22422025-12-16T03:00:39.208ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
22432025-12-16T03:00:39.208ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 19 reason = Timeout
22442025-12-16T03:00:39.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22452025-12-16T03:00:39.208ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
22462025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
22472025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 4 reason = Timeout
22482025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 8 reason = Timeout
22492025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22502025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 4 reason = Timeout
22512025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
22522025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
22532025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
22542025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 6 reason = Timeout
22552025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
22562025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 4 reason = Timeout
22572025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
22582025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 6 reason = Timeout
22592025-12-16T03:00:39.209ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
22602025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
22612025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22622025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 4 reason = Timeout
22632025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 4 reason = Timeout
22642025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22652025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22662025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 4 reason = Timeout
22672025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
22682025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 10 reason = Timeout
22692025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
22702025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
22712025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
22722025-12-16T03:00:39.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
22732025-12-16T03:00:39.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
22742025-12-16T03:00:39.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22752025-12-16T03:00:39.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
22762025-12-16T03:00:39.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
22772025-12-16T03:00:39.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22782025-12-16T03:00:39.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
22792025-12-16T03:00:39.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
22802025-12-16T03:00:39.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22812025-12-16T03:00:39.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22822025-12-16T03:00:39.211ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22832025-12-16T03:00:39.212ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22842025-12-16T03:00:39.213ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
22852025-12-16T03:00:39.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
22862025-12-16T03:00:39.214ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22872025-12-16T03:00:39.215ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22882025-12-16T03:00:39.215ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22892025-12-16T03:00:39.216ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22902025-12-16T03:00:39.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
22912025-12-16T03:00:39.216ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
22922025-12-16T03:00:39.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
22932025-12-16T03:00:39.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
22942025-12-16T03:00:39.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
22952025-12-16T03:00:39.218ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
22962025-12-16T03:00:39.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22972025-12-16T03:00:39.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
22982025-12-16T03:00:39.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
22992025-12-16T03:00:39.220ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
23002025-12-16T03:00:39.220ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23012025-12-16T03:00:39.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
23022025-12-16T03:00:39.220ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
23032025-12-16T03:00:39.220ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23042025-12-16T03:00:39.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
23052025-12-16T03:00:39.220ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23062025-12-16T03:00:39.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23072025-12-16T03:00:39.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23082025-12-16T03:00:39.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
23092025-12-16T03:00:39.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
23102025-12-16T03:00:39.221ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
23112025-12-16T03:00:39.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
23122025-12-16T03:00:39.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23132025-12-16T03:00:39.222ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
23142025-12-16T03:00:39.222ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
23152025-12-16T03:00:39.222ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
23162025-12-16T03:00:39.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
23172025-12-16T03:00:39.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
23182025-12-16T03:00:39.222ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 993293d3-f4d1-4544-a5c6-b0c29460233f original_time_started = 2025-12-16 03:00:10.647828 UTC
23192025-12-16T03:00:39.222ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 14.074449ms iteration = 10 status = {"Loaded":{"collection_id":"993293d3-f4d1-4544-a5c6-b0c29460233f","time_loaded":"2025-12-16T03:00:39.208886467Z","time_started":"2025-12-16T03:00:10.647828Z"}}
23202025-12-16T03:00:39.222ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23212025-12-16T03:00:39.223ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
23222025-12-16T03:00:39.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
23232025-12-16T03:00:39.224ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
23242025-12-16T03:00:39.224ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
23252025-12-16T03:00:39.224ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
23262025-12-16T03:00:39.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
23272025-12-16T03:00:39.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
23282025-12-16T03:00:39.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23292025-12-16T03:00:39.226ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
23302025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
23312025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
23322025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 4 reason = Timeout
23332025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 15.04\u{b5}s iteration = 4 status = {"disabled":true,"errors":[],"sps":[]}
23342025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 4 reason = Timeout
23352025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
23362025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 4 reason = Timeout
23372025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
23382025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 4 reason = Timeout
23392025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
23402025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 4 reason = Timeout
23412025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
23422025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 7 reason = Timeout
23432025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
23442025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 4 reason = Timeout
23452025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
23462025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 7 reason = Timeout
23472025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
23482025-12-16T03:00:39.226ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1046, tv_nsec: 843803951 } })
23492025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
23502025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
23512025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
23522025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
23532025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
23542025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
23552025-12-16T03:00:39.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
23562025-12-16T03:00:39.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
23572025-12-16T03:00:39.227ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
23582025-12-16T03:00:39.227ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
23592025-12-16T03:00:39.227ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
23602025-12-16T03:00:39.227ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
23612025-12-16T03:00:39.227ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 18.439923ms iteration = 19 status = {"config_updated":false}
23622025-12-16T03:00:39.227ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1046, tv_nsec: 844835824 } })
23632025-12-16T03:00:39.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
23642025-12-16T03:00:39.227ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
23652025-12-16T03:00:39.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
23662025-12-16T03:00:39.229ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 3.086969ms iteration = 4 status = {"errors":[],"finish_invoked_ok":[]}
23672025-12-16T03:00:39.229ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23682025-12-16T03:00:39.230ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23692025-12-16T03:00:39.231ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
23702025-12-16T03:00:39.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23712025-12-16T03:00:39.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
23722025-12-16T03:00:39.232ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
23732025-12-16T03:00:39.233ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
23742025-12-16T03:00:39.233ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
23752025-12-16T03:00:39.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
23762025-12-16T03:00:39.235ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 9.550701ms iteration = 7 status = "NoSitrep"
23772025-12-16T03:00:39.236ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
23782025-12-16T03:00:39.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
23792025-12-16T03:00:39.237ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
23802025-12-16T03:00:39.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
23812025-12-16T03:00:39.240ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23822025-12-16T03:00:39.242ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
23832025-12-16T03:00:39.246ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
23842025-12-16T03:00:39.246ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 20.164405ms iteration = 7 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
23852025-12-16T03:00:39.263ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
23862025-12-16T03:00:39.263ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 54.751804ms iteration = 10 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
23872025-12-16T03:00:39.272ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
23882025-12-16T03:00:39.272ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
23892025-12-16T03:00:39.272ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e541281-4cc3-4f97-a582-47820911e06b", "content-length": "26", "date": "Tue, 16 Dec 2025 03:00:39 GMT"} })
23902025-12-16T03:00:39.272ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
23912025-12-16T03:00:39.274ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
23922025-12-16T03:00:39.274ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1046, tv_nsec: 892027942 } })
23932025-12-16T03:00:39.277ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
23942025-12-16T03:00:39.278ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23952025-12-16T03:00:39.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23962025-12-16T03:00:39.279ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 111.696905ms iteration = 4 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
23972025-12-16T03:00:39.279ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
23982025-12-16T03:00:39.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
23992025-12-16T03:00:39.281ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 71.647578ms iteration = 10 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
24002025-12-16T03:00:39.281ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24012025-12-16T03:00:39.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
24022025-12-16T03:00:39.285ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
24032025-12-16T03:00:39.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
24042025-12-16T03:00:39.288ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24052025-12-16T03:00:39.288ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 62.227979ms iteration = 4 status = {"errors":[],"garbage_collect_requested":[]}
24062025-12-16T03:00:39.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
24072025-12-16T03:00:39.293ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24082025-12-16T03:00:39.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24092025-12-16T03:00:39.296ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
24102025-12-16T03:00:39.296ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 86.939751ms iteration = 4 status = 0
24112025-12-16T03:00:39.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
24122025-12-16T03:00:39.296ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 87.397258ms iteration = 4 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
24132025-12-16T03:00:39.298ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
24142025-12-16T03:00:39.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
24152025-12-16T03:00:39.300ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24162025-12-16T03:00:39.302ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
24172025-12-16T03:00:39.302ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24182025-12-16T03:00:39.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24192025-12-16T03:00:39.304ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
24202025-12-16T03:00:39.304ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
24212025-12-16T03:00:39.307ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a41249e-334f-4733-877f-68ccbad9bec5", "content-length": "26", "date": "Tue, 16 Dec 2025 03:00:39 GMT"} })
24222025-12-16T03:00:39.307ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
24232025-12-16T03:00:39.307ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
24242025-12-16T03:00:39.307ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 97.449025ms iteration = 4 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
24252025-12-16T03:00:39.307ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
24262025-12-16T03:00:39.308ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 82.11878ms iteration = 4 status = {"errors":[],"requests_created_ok":[]}
24272025-12-16T03:00:39.312ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
24282025-12-16T03:00:39.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
24292025-12-16T03:00:39.314ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 88.137587ms iteration = 4 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
24302025-12-16T03:00:39.323ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
24312025-12-16T03:00:39.323ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
24322025-12-16T03:00:39.323ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 114.328908ms iteration = 4 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
24332025-12-16T03:00:39.337ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 169.917043ms iteration = 4 status = {}
24342025-12-16T03:00:39.344ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
24352025-12-16T03:00:39.344ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 134.710887ms iteration = 6 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
24362025-12-16T03:00:39.359ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 132.806133ms iteration = 4 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
24372025-12-16T03:00:39.361ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1046, tv_nsec: 978700840 } })
24382025-12-16T03:00:39.367ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 158.448488ms iteration = 8 status = {"physical_disks_added":0}
24392025-12-16T03:00:39.381ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
24402025-12-16T03:00:39.381ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
24412025-12-16T03:00:39.382ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "67f88cbe-9438-4d17-92de-048a59229f30", "content-length": "372", "date": "Tue, 16 Dec 2025 03:00:39 GMT"} })
24422025-12-16T03:00:39.382ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24432025-12-16T03:00:39.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24442025-12-16T03:00:39.392ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
24452025-12-16T03:00:39.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
24462025-12-16T03:00:39.401ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1047, tv_nsec: 19182733 } })
24472025-12-16T03:00:39.434ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24482025-12-16T03:00:39.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24492025-12-16T03:00:39.437ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
24502025-12-16T03:00:39.437ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24512025-12-16T03:00:39.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24522025-12-16T03:00:39.438ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
24532025-12-16T03:00:39.438ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
24542025-12-16T03:00:39.440ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "54598b41-37f5-4c3b-abfd-b747a10a25cc", "date": "Tue, 16 Dec 2025 03:00:39 GMT"} })
24552025-12-16T03:00:39.440ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24562025-12-16T03:00:39.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24572025-12-16T03:00:39.443ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24582025-12-16T03:00:39.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24592025-12-16T03:00:39.446ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24602025-12-16T03:00:39.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24612025-12-16T03:00:39.449ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
24622025-12-16T03:00:39.449ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
24632025-12-16T03:00:39.450ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "f0922437-6123-4513-ac57-95424ae17f41", "date": "Tue, 16 Dec 2025 03:00:39 GMT"} })
24642025-12-16T03:00:39.451ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 241.833543ms iteration = 6 status = {}
24652025-12-16T03:00:39.451ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
24662025-12-16T03:00:39.451ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
24672025-12-16T03:00:39.451ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
24682025-12-16T03:00:39.451ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
24692025-12-16T03:00:39.451ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
24702025-12-16T03:00:39.452ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
24712025-12-16T03:00:39.452ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
24722025-12-16T03:00:39.452ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
24732025-12-16T03:00:39.452ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
24742025-12-16T03:00:39.452ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
24752025-12-16T03:00:39.452ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
24762025-12-16T03:00:39.452ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24772025-12-16T03:00:39.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24782025-12-16T03:00:39.454ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
24792025-12-16T03:00:39.454ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
24802025-12-16T03:00:39.527ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
24812025-12-16T03:00:39.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
24822025-12-16T03:00:39.529ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24832025-12-16T03:00:39.531ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
24842025-12-16T03:00:39.531ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
24852025-12-16T03:00:39.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
24862025-12-16T03:00:39.533ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24872025-12-16T03:00:39.535ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
24882025-12-16T03:00:39.535ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
24892025-12-16T03:00:39.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
24902025-12-16T03:00:39.567ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
24912025-12-16T03:00:39.570ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a523da59-6eef-480d-994f-7efa653b7fb9", "content-length": "691", "date": "Tue, 16 Dec 2025 03:00:39 GMT"} })
24922025-12-16T03:00:39.570ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
24932025-12-16T03:00:39.570ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
24942025-12-16T03:00:39.570ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
24952025-12-16T03:00:39.573ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 363.602386ms iteration = 4 status = {}
24962025-12-16T03:00:39.697ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "c4715ffa-1253-422d-8588-710e78162f8a", "date": "Tue, 16 Dec 2025 03:00:39 GMT"} })
24972025-12-16T03:00:39.697ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
24982025-12-16T03:00:39.697ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
24992025-12-16T03:00:39.697ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0184688-559a-4568-8809-99320515295c", "content-length": "682", "date": "Tue, 16 Dec 2025 03:00:39 GMT"} })
25002025-12-16T03:00:39.697ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25012025-12-16T03:00:39.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25022025-12-16T03:00:39.713ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25032025-12-16T03:00:39.713ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25042025-12-16T03:00:39.715ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25052025-12-16T03:00:39.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25062025-12-16T03:00:39.718ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25072025-12-16T03:00:39.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25082025-12-16T03:00:39.725ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 515.494661ms iteration = 4 status = {}
25092025-12-16T03:00:41.060ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 295 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 9a3943f8-ef19-4ff5-b401-da56cbe24813 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
25102025-12-16T03:00:44.209ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 20 reason = Timeout
25112025-12-16T03:00:44.209ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
25122025-12-16T03:00:44.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
25132025-12-16T03:00:44.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25142025-12-16T03:00:44.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
25152025-12-16T03:00:44.214ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
25162025-12-16T03:00:44.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
25172025-12-16T03:00:44.216ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
25182025-12-16T03:00:44.216ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.265262ms iteration = 20 status = {"config_updated":false}
25192025-12-16T03:00:49.208ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 11 reason = Timeout
25202025-12-16T03:00:49.208ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 21 reason = Timeout
25212025-12-16T03:00:49.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25222025-12-16T03:00:49.208ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
25232025-12-16T03:00:49.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
25242025-12-16T03:00:49.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
25252025-12-16T03:00:49.209ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 11 reason = Timeout
25262025-12-16T03:00:49.209ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25272025-12-16T03:00:49.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25282025-12-16T03:00:49.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25292025-12-16T03:00:49.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25302025-12-16T03:00:49.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25312025-12-16T03:00:49.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25322025-12-16T03:00:49.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
25332025-12-16T03:00:49.215ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
25342025-12-16T03:00:49.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
25352025-12-16T03:00:49.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
25362025-12-16T03:00:49.216ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
25372025-12-16T03:00:49.216ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.553745ms iteration = 21 status = {"config_updated":false}
25382025-12-16T03:00:49.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25392025-12-16T03:00:49.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
25402025-12-16T03:00:49.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25412025-12-16T03:00:49.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
25422025-12-16T03:00:49.221ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25432025-12-16T03:00:49.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
25442025-12-16T03:00:49.241ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
25452025-12-16T03:00:49.242ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 33.107379ms iteration = 11 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
25462025-12-16T03:00:49.255ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25472025-12-16T03:00:49.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
25482025-12-16T03:00:49.297ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 87.878653ms iteration = 11 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
25492025-12-16T03:00:51.060ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 310 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 3d051aa8-d11e-4053-a730-932dbad21e85 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
25502025-12-16T03:00:54.209ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 11 reason = Timeout
25512025-12-16T03:00:54.209ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 22 reason = Timeout
25522025-12-16T03:00:54.209ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
25532025-12-16T03:00:54.209ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
25542025-12-16T03:00:54.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
25552025-12-16T03:00:54.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
25562025-12-16T03:00:54.211ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25572025-12-16T03:00:54.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25582025-12-16T03:00:54.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
25592025-12-16T03:00:54.215ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
25602025-12-16T03:00:54.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
25612025-12-16T03:00:54.215ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
25622025-12-16T03:00:54.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
25632025-12-16T03:00:54.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
25642025-12-16T03:00:54.218ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
25652025-12-16T03:00:54.218ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.991034ms iteration = 22 status = {"config_updated":false}
25662025-12-16T03:00:54.218ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 993293d3-f4d1-4544-a5c6-b0c29460233f original_time_started = 2025-12-16 03:00:10.647828 UTC
25672025-12-16T03:00:54.218ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 9.035114ms iteration = 11 status = {"Loaded":{"collection_id":"993293d3-f4d1-4544-a5c6-b0c29460233f","time_loaded":"2025-12-16T03:00:54.209171938Z","time_started":"2025-12-16T03:00:10.647828Z"}}
25682025-12-16T03:00:54.226ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 8 reason = Timeout
25692025-12-16T03:00:54.226ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 8 reason = Timeout
25702025-12-16T03:00:54.226ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
25712025-12-16T03:00:54.226ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
25722025-12-16T03:00:54.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
25732025-12-16T03:00:54.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
25742025-12-16T03:00:54.227ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
25752025-12-16T03:00:54.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
25762025-12-16T03:00:54.228ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25772025-12-16T03:00:54.228ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25782025-12-16T03:00:54.230ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
25792025-12-16T03:00:54.230ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
25802025-12-16T03:00:54.230ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
25812025-12-16T03:00:54.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
25822025-12-16T03:00:54.232ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
25832025-12-16T03:00:54.232ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.225889ms iteration = 8 status = "NoSitrep"
25842025-12-16T03:00:54.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
25852025-12-16T03:00:54.233ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
25862025-12-16T03:00:54.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
25872025-12-16T03:00:54.234ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25882025-12-16T03:00:54.236ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
25892025-12-16T03:00:54.236ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
25902025-12-16T03:00:54.236ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.454702ms iteration = 8 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
25912025-12-16T03:00:59.209ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 12 reason = Timeout
25922025-12-16T03:00:59.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25932025-12-16T03:00:59.209ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 23 reason = Timeout
25942025-12-16T03:00:59.209ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
25952025-12-16T03:00:59.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
25962025-12-16T03:00:59.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
25972025-12-16T03:00:59.209ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 12 reason = Timeout
25982025-12-16T03:00:59.209ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
25992025-12-16T03:00:59.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26002025-12-16T03:00:59.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26012025-12-16T03:00:59.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26022025-12-16T03:00:59.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
26032025-12-16T03:00:59.215ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
26042025-12-16T03:00:59.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
26052025-12-16T03:00:59.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
26062025-12-16T03:00:59.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26072025-12-16T03:00:59.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
26082025-12-16T03:00:59.216ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
26092025-12-16T03:00:59.216ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.811399ms iteration = 23 status = {"config_updated":false}
26102025-12-16T03:00:59.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26112025-12-16T03:00:59.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
26122025-12-16T03:00:59.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26132025-12-16T03:00:59.223ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
26142025-12-16T03:00:59.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26152025-12-16T03:00:59.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
26162025-12-16T03:00:59.238ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
26172025-12-16T03:00:59.238ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.696386ms iteration = 12 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
26182025-12-16T03:00:59.248ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26192025-12-16T03:00:59.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26202025-12-16T03:00:59.285ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 76.098814ms iteration = 12 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
26212025-12-16T03:01:01.060ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 267 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 6bffb63e-8871-452b-8d6e-9a98d2e48377 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
26222025-12-16T03:01:04.209ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 24 reason = Timeout
26232025-12-16T03:01:04.209ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
26242025-12-16T03:01:04.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
26252025-12-16T03:01:04.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26262025-12-16T03:01:04.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
26272025-12-16T03:01:04.215ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
26282025-12-16T03:01:04.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
26292025-12-16T03:01:04.217ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
26302025-12-16T03:01:04.217ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.736021ms iteration = 24 status = {"config_updated":false}
26312025-12-16T03:01:09.167ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 5 reason = Timeout
26322025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 3 reason = Timeout
26332025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
26342025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 5 reason = Timeout
26352025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
26362025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 3 reason = Timeout
26372025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 3 reason = Timeout
26382025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 4 reason = Timeout
26392025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
26402025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
26412025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 5 reason = Timeout
26422025-12-16T03:01:09.168ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
26432025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 3 reason = Timeout
26442025-12-16T03:01:09.168ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 25248e48-0801-41d1-9c12-2383bbe16c17 step component = ExternalNetworking step id = Ensure
26452025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 5 reason = Timeout
26462025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
26472025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
26482025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 3 reason = Timeout
26492025-12-16T03:01:09.168ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-12-16 02:51:09.168122835 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
26502025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
26512025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 3 reason = Timeout
26522025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
26532025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
26542025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 5 reason = Timeout
26552025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
26562025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
26572025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
26582025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
26592025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
26602025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
26612025-12-16T03:01:09.168ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("49a51a59-9919-48ae-8182-4070a8430160.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6af60386-8771-4401-9d4d-e93774e519f0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 1076, tv_nsec: 785746975 } })
26622025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
26632025-12-16T03:01:09.168ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1a45cd23-3d0a-43eb-adce-62c7b712baf1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1076, tv_nsec: 785835937 } })
26642025-12-16T03:01:09.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
26652025-12-16T03:01:09.169ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
26662025-12-16T03:01:09.169ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.300776ms iteration = 3 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
26672025-12-16T03:01:09.169ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
26682025-12-16T03:01:09.169ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.60392ms iteration = 3 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
26692025-12-16T03:01:09.169ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
26702025-12-16T03:01:09.170ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
26712025-12-16T03:01:09.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
26722025-12-16T03:01:09.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
26732025-12-16T03:01:09.171ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1076, tv_nsec: 788501470 } })
26742025-12-16T03:01:09.171ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.837719ms iteration = 5 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
26752025-12-16T03:01:09.172ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-12-16 02:51:09.168122835 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
26762025-12-16T03:01:09.172ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.521928ms iteration = 3 status = {"expiration":"2025-12-16T02:51:09.168122835Z","pruned":[]}
26772025-12-16T03:01:09.172ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1076, tv_nsec: 789675505 } })
26782025-12-16T03:01:09.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
26792025-12-16T03:01:09.173ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
26802025-12-16T03:01:09.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
26812025-12-16T03:01:09.173ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1076, tv_nsec: 790456665 } })
26822025-12-16T03:01:09.175ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
26832025-12-16T03:01:09.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
26842025-12-16T03:01:09.180ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26852025-12-16T03:01:09.185ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
26862025-12-16T03:01:09.194ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
26872025-12-16T03:01:09.198ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
26882025-12-16T03:01:09.201ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
26892025-12-16T03:01:09.205ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
26902025-12-16T03:01:09.208ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
26912025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 9 reason = Timeout
26922025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 13 reason = Timeout
26932025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
26942025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26952025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 25 reason = Timeout
26962025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
26972025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 12 reason = Timeout
26982025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 3 reason = Timeout
26992025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
27002025-12-16T03:01:09.209ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
27012025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 5 reason = Timeout
27022025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
27032025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
27042025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 7 reason = Timeout
27052025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
27062025-12-16T03:01:09.209ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
27072025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27082025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 7 reason = Timeout
27092025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
27102025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 9 reason = Timeout
27112025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
27122025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 5 reason = Timeout
27132025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
27142025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 5 reason = Timeout
27152025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27162025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 13 reason = Timeout
27172025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
27182025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 5 reason = Timeout
27192025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
27202025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 5 reason = Timeout
27212025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
27222025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 5 reason = Timeout
27232025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
27242025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 3 reason = Timeout
27252025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
27262025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 3 reason = Timeout
27272025-12-16T03:01:09.209ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
27282025-12-16T03:01:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27292025-12-16T03:01:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
27302025-12-16T03:01:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
27312025-12-16T03:01:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
27322025-12-16T03:01:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
27332025-12-16T03:01:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
27342025-12-16T03:01:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
27352025-12-16T03:01:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
27362025-12-16T03:01:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
27372025-12-16T03:01:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
27382025-12-16T03:01:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27392025-12-16T03:01:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
27402025-12-16T03:01:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
27412025-12-16T03:01:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
27422025-12-16T03:01:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
27432025-12-16T03:01:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
27442025-12-16T03:01:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27452025-12-16T03:01:09.212ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27462025-12-16T03:01:09.213ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
27472025-12-16T03:01:09.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27482025-12-16T03:01:09.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27492025-12-16T03:01:09.215ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27502025-12-16T03:01:09.217ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27512025-12-16T03:01:09.217ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27522025-12-16T03:01:09.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27532025-12-16T03:01:09.218ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27542025-12-16T03:01:09.219ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
27552025-12-16T03:01:09.219ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
27562025-12-16T03:01:09.219ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
27572025-12-16T03:01:09.219ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27582025-12-16T03:01:09.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
27592025-12-16T03:01:09.221ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
27602025-12-16T03:01:09.221ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27612025-12-16T03:01:09.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
27622025-12-16T03:01:09.221ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
27632025-12-16T03:01:09.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27642025-12-16T03:01:09.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
27652025-12-16T03:01:09.222ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27662025-12-16T03:01:09.223ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
27672025-12-16T03:01:09.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
27682025-12-16T03:01:09.223ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
27692025-12-16T03:01:09.223ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
27702025-12-16T03:01:09.223ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 993293d3-f4d1-4544-a5c6-b0c29460233f original_time_started = 2025-12-16 03:00:10.647828 UTC
27712025-12-16T03:01:09.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
27722025-12-16T03:01:09.223ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 13.856416ms iteration = 12 status = {"Loaded":{"collection_id":"993293d3-f4d1-4544-a5c6-b0c29460233f","time_loaded":"2025-12-16T03:01:09.209631112Z","time_started":"2025-12-16T03:00:10.647828Z"}}
27732025-12-16T03:01:09.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
27742025-12-16T03:01:09.223ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
27752025-12-16T03:01:09.224ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27762025-12-16T03:01:09.224ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
27772025-12-16T03:01:09.224ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
27782025-12-16T03:01:09.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
27792025-12-16T03:01:09.226ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
27802025-12-16T03:01:09.226ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
27812025-12-16T03:01:09.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
27822025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 10 reason = Timeout
27832025-12-16T03:01:09.227ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
27842025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 5 reason = Timeout
27852025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
27862025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 25.37\u{b5}s iteration = 10 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
27872025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 5 reason = Timeout
27882025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
27892025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 3 reason = Timeout
27902025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
27912025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 5 reason = Timeout
27922025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
27932025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 9 reason = Timeout
27942025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
27952025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 3 reason = Timeout
27962025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
27972025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 5 reason = Timeout
27982025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
27992025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 9 reason = Timeout
28002025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
28012025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 5 reason = Timeout
28022025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
28032025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 3 reason = Timeout
28042025-12-16T03:01:09.227ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
28052025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
28062025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 5 reason = Timeout
28072025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 12.171\u{b5}s iteration = 5 status = {"disabled":true,"errors":[],"sps":[]}
28082025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 3 reason = Timeout
28092025-12-16T03:01:09.227ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
28102025-12-16T03:01:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
28112025-12-16T03:01:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
28122025-12-16T03:01:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
28132025-12-16T03:01:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
28142025-12-16T03:01:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
28152025-12-16T03:01:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
28162025-12-16T03:01:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
28172025-12-16T03:01:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
28182025-12-16T03:01:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
28192025-12-16T03:01:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
28202025-12-16T03:01:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
28212025-12-16T03:01:09.229ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
28222025-12-16T03:01:09.229ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
28232025-12-16T03:01:09.229ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
28242025-12-16T03:01:09.229ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
28252025-12-16T03:01:09.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
28262025-12-16T03:01:09.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
28272025-12-16T03:01:09.231ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
28282025-12-16T03:01:09.236ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28292025-12-16T03:01:09.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28302025-12-16T03:01:09.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28312025-12-16T03:01:09.237ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
28322025-12-16T03:01:09.237ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 28.252928ms iteration = 25 status = {"config_updated":false}
28332025-12-16T03:01:09.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
28342025-12-16T03:01:09.240ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
28352025-12-16T03:01:09.250ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
28362025-12-16T03:01:09.257ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
28372025-12-16T03:01:09.258ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 25248e48-0801-41d1-9c12-2383bbe16c17 step component = SledList step id = Fetch
28382025-12-16T03:01:09.258ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
28392025-12-16T03:01:09.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
28402025-12-16T03:01:09.279ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28412025-12-16T03:01:09.280ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
28422025-12-16T03:01:09.281ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
28432025-12-16T03:01:09.281ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
28442025-12-16T03:01:09.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
28452025-12-16T03:01:09.289ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 61.441999ms iteration = 3 status = {"errors":[],"found_port_ok":[]}
28462025-12-16T03:01:09.294ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28472025-12-16T03:01:09.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
28482025-12-16T03:01:09.301ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28492025-12-16T03:01:09.303ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28502025-12-16T03:01:09.306ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28512025-12-16T03:01:09.307ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
28522025-12-16T03:01:09.308ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
28532025-12-16T03:01:09.308ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
28542025-12-16T03:01:09.308ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
28552025-12-16T03:01:09.308ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 81.096627ms iteration = 3 status = {"by_rx":{},"error":null}
28562025-12-16T03:01:09.309ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
28572025-12-16T03:01:09.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
28582025-12-16T03:01:09.310ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
28592025-12-16T03:01:09.310ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
28602025-12-16T03:01:09.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
28612025-12-16T03:01:09.310ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
28622025-12-16T03:01:09.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
28632025-12-16T03:01:09.312ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28642025-12-16T03:01:09.316ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
28652025-12-16T03:01:09.316ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
28662025-12-16T03:01:09.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
28672025-12-16T03:01:09.320ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28682025-12-16T03:01:09.320ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
28692025-12-16T03:01:09.320ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
28702025-12-16T03:01:09.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
28712025-12-16T03:01:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
28722025-12-16T03:01:09.321ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28732025-12-16T03:01:09.322ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 113.045303ms iteration = 5 status = 0
28742025-12-16T03:01:09.323ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28752025-12-16T03:01:09.324ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
28762025-12-16T03:01:09.324ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
28772025-12-16T03:01:09.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
28782025-12-16T03:01:09.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
28792025-12-16T03:01:09.326ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
28802025-12-16T03:01:09.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
28812025-12-16T03:01:09.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28822025-12-16T03:01:09.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
28832025-12-16T03:01:09.328ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28842025-12-16T03:01:09.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28852025-12-16T03:01:09.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28862025-12-16T03:01:09.329ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
28872025-12-16T03:01:09.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
28882025-12-16T03:01:09.331ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28892025-12-16T03:01:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
28902025-12-16T03:01:09.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
28912025-12-16T03:01:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
28922025-12-16T03:01:09.333ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
28932025-12-16T03:01:09.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
28942025-12-16T03:01:09.334ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
28952025-12-16T03:01:09.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
28962025-12-16T03:01:09.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
28972025-12-16T03:01:09.334ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
28982025-12-16T03:01:09.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
28992025-12-16T03:01:09.336ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
29002025-12-16T03:01:09.336ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
29012025-12-16T03:01:09.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
29022025-12-16T03:01:09.338ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 110.903236ms iteration = 9 status = "NoSitrep"
29032025-12-16T03:01:09.339ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1076, tv_nsec: 956356187 } })
29042025-12-16T03:01:09.339ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
29052025-12-16T03:01:09.340ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1076, tv_nsec: 957299709 } })
29062025-12-16T03:01:09.341ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29072025-12-16T03:01:09.341ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
29082025-12-16T03:01:09.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
29092025-12-16T03:01:09.343ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29102025-12-16T03:01:09.343ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
29112025-12-16T03:01:09.343ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
29122025-12-16T03:01:09.343ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 133.415241ms iteration = 3 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
29132025-12-16T03:01:09.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
29142025-12-16T03:01:09.344ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
29152025-12-16T03:01:09.344ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
29162025-12-16T03:01:09.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
29172025-12-16T03:01:09.347ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 120.040491ms iteration = 5 status = {"errors":[],"finish_invoked_ok":[]}
29182025-12-16T03:01:09.348ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
29192025-12-16T03:01:09.348ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 138.277642ms iteration = 5 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
29202025-12-16T03:01:09.348ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 138.83437ms iteration = 13 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
29212025-12-16T03:01:09.348ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 121.132995ms iteration = 3 status = {"dispatched":[],"errors":[],"glob_version":"212.0.0","globs_reprocessed":{},"no_receivers":[]}
29222025-12-16T03:01:09.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29232025-12-16T03:01:09.350ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29242025-12-16T03:01:09.351ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
29252025-12-16T03:01:09.351ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
29262025-12-16T03:01:09.351ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 141.864198ms iteration = 3 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
29272025-12-16T03:01:09.354ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29282025-12-16T03:01:09.354ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
29292025-12-16T03:01:09.354ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29302025-12-16T03:01:09.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29312025-12-16T03:01:09.355ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
29322025-12-16T03:01:09.355ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
29332025-12-16T03:01:09.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
29342025-12-16T03:01:09.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
29352025-12-16T03:01:09.360ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
29362025-12-16T03:01:09.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
29372025-12-16T03:01:09.361ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 25248e48-0801-41d1-9c12-2383bbe16c17 step component = DeployNexusRecords step id = Ensure
29382025-12-16T03:01:09.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29392025-12-16T03:01:09.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29402025-12-16T03:01:09.364ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
29412025-12-16T03:01:09.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
29422025-12-16T03:01:09.366ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29432025-12-16T03:01:09.367ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29442025-12-16T03:01:09.369ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
29452025-12-16T03:01:09.369ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29462025-12-16T03:01:09.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29472025-12-16T03:01:09.371ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
29482025-12-16T03:01:09.371ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
29492025-12-16T03:01:09.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
29502025-12-16T03:01:09.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
29512025-12-16T03:01:09.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
29522025-12-16T03:01:09.388ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
29532025-12-16T03:01:09.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
29542025-12-16T03:01:09.388ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 161.099081ms iteration = 5 status = {"errors":[],"garbage_collect_requested":[]}
29552025-12-16T03:01:09.388ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
29562025-12-16T03:01:09.389ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
29572025-12-16T03:01:09.389ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
29582025-12-16T03:01:09.389ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
29592025-12-16T03:01:09.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
29602025-12-16T03:01:09.391ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
29612025-12-16T03:01:09.391ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
29622025-12-16T03:01:09.391ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
29632025-12-16T03:01:09.391ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29642025-12-16T03:01:09.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29652025-12-16T03:01:09.391ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 163.845876ms iteration = 5 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
29662025-12-16T03:01:09.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
29672025-12-16T03:01:09.392ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29682025-12-16T03:01:09.393ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29692025-12-16T03:01:09.394ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
29702025-12-16T03:01:09.394ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 184.929423ms iteration = 13 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
29712025-12-16T03:01:09.397ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
29722025-12-16T03:01:09.399ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
29732025-12-16T03:01:09.399ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
29742025-12-16T03:01:09.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
29752025-12-16T03:01:09.400ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
29762025-12-16T03:01:09.400ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29772025-12-16T03:01:09.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
29782025-12-16T03:01:09.403ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
29792025-12-16T03:01:09.403ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
29802025-12-16T03:01:09.403ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29812025-12-16T03:01:09.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
29822025-12-16T03:01:09.404ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 176.972413ms iteration = 5 status = {"errors":[],"requests_created_ok":[]}
29832025-12-16T03:01:09.412ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
29842025-12-16T03:01:09.413ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29852025-12-16T03:01:09.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
29862025-12-16T03:01:09.418ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
29872025-12-16T03:01:09.418ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
29882025-12-16T03:01:09.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
29892025-12-16T03:01:09.420ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29902025-12-16T03:01:09.422ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
29912025-12-16T03:01:09.422ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
29922025-12-16T03:01:09.424ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
29932025-12-16T03:01:09.424ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
29942025-12-16T03:01:09.424ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-12-16 02:59:18.154440 UTC dns_group = external generation = 2
29952025-12-16T03:01:09.424ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 256.550701ms iteration = 3 status = {"generation":2}
29962025-12-16T03:01:09.424ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 256.809434ms iteration = 5 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
29972025-12-16T03:01:09.430ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
29982025-12-16T03:01:09.430ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
29992025-12-16T03:01:09.430ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
30002025-12-16T03:01:09.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
30012025-12-16T03:01:09.431ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 221.550806ms iteration = 5 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
30022025-12-16T03:01:09.433ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
30032025-12-16T03:01:09.433ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 223.679644ms iteration = 7 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
30042025-12-16T03:01:09.436ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
30052025-12-16T03:01:09.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
30062025-12-16T03:01:09.437ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
30072025-12-16T03:01:09.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
30082025-12-16T03:01:09.441ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30092025-12-16T03:01:09.444ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30102025-12-16T03:01:09.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
30112025-12-16T03:01:09.444ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
30122025-12-16T03:01:09.446ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
30132025-12-16T03:01:09.446ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 218.69283ms iteration = 9 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
30142025-12-16T03:01:09.447ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30152025-12-16T03:01:09.448ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
30162025-12-16T03:01:09.448ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
30172025-12-16T03:01:09.449ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c236ce5-a0b4-449d-9f25-bc5f40116e89", "content-length": "26", "date": "Tue, 16 Dec 2025 03:01:09 GMT"} })
30182025-12-16T03:01:09.449ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
30192025-12-16T03:01:09.450ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
30202025-12-16T03:01:09.451ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1077, tv_nsec: 68147843 } })
30212025-12-16T03:01:09.451ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
30222025-12-16T03:01:09.451ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
30232025-12-16T03:01:09.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
30242025-12-16T03:01:09.480ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
30252025-12-16T03:01:09.480ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
30262025-12-16T03:01:09.481ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30272025-12-16T03:01:09.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
30282025-12-16T03:01:09.485ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30292025-12-16T03:01:09.487ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
30302025-12-16T03:01:09.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
30312025-12-16T03:01:09.490ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 25248e48-0801-41d1-9c12-2383bbe16c17 step component = SledAgent step id = Ensure
30322025-12-16T03:01:09.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30332025-12-16T03:01:09.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30342025-12-16T03:01:09.491ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 263.594739ms iteration = 5 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
30352025-12-16T03:01:09.491ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
30362025-12-16T03:01:09.491ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
30372025-12-16T03:01:09.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
30382025-12-16T03:01:09.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30392025-12-16T03:01:09.495ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
30402025-12-16T03:01:09.495ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
30412025-12-16T03:01:09.495ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-12-16 02:59:16.791641 UTC dns_group = internal generation = 1
30422025-12-16T03:01:09.495ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 327.705652ms iteration = 3 status = {"generation":1}
30432025-12-16T03:01:09.495ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
30442025-12-16T03:01:09.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30452025-12-16T03:01:09.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30462025-12-16T03:01:09.497ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30472025-12-16T03:01:09.498ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
30482025-12-16T03:01:09.498ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
30492025-12-16T03:01:09.498ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6d849df-8267-4be1-bccc-edcb3d38795d", "content-length": "26", "date": "Tue, 16 Dec 2025 03:01:09 GMT"} })
30502025-12-16T03:01:09.498ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
30512025-12-16T03:01:09.501ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
30522025-12-16T03:01:09.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30532025-12-16T03:01:09.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30542025-12-16T03:01:09.502ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
30552025-12-16T03:01:09.505ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
30562025-12-16T03:01:09.506ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "ee43631b-0494-46dd-a730-2647cc305c84", "date": "Tue, 16 Dec 2025 03:01:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
30572025-12-16T03:01:09.506ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
30582025-12-16T03:01:09.507ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
30592025-12-16T03:01:09.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
30602025-12-16T03:01:09.508ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "4426c808-68a8-40a5-8b69-0cb933c3fd3c", "date": "Tue, 16 Dec 2025 03:01:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
30612025-12-16T03:01:09.508ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
30622025-12-16T03:01:09.508ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30632025-12-16T03:01:09.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
30642025-12-16T03:01:09.508ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
30652025-12-16T03:01:09.508ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
30662025-12-16T03:01:09.508ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 298.991628ms iteration = 5 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
30672025-12-16T03:01:09.510ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
30682025-12-16T03:01:09.510ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 342.247986ms iteration = 5 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
30692025-12-16T03:01:09.519ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-12-16T02:59:05.020237Z, time_modified: 2025-12-16T02:59:05.020237Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }) }
30702025-12-16T03:01:09.519ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 351.423172ms iteration = 3 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]}},"default_endpoint":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
30712025-12-16T03:01:09.527ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
30722025-12-16T03:01:09.558ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
30732025-12-16T03:01:09.558ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
30742025-12-16T03:01:09.559ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ffada71-fe0a-4982-beff-872679bc453e", "content-length": "372", "date": "Tue, 16 Dec 2025 03:01:09 GMT"} })
30752025-12-16T03:01:09.559ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30762025-12-16T03:01:09.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30772025-12-16T03:01:09.568ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30782025-12-16T03:01:09.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30792025-12-16T03:01:09.569ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30802025-12-16T03:01:09.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30812025-12-16T03:01:09.571ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30822025-12-16T03:01:09.577ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
30832025-12-16T03:01:09.577ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30842025-12-16T03:01:09.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30852025-12-16T03:01:09.578ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 368.443427ms iteration = 9 status = {"physical_disks_added":0}
30862025-12-16T03:01:09.580ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30872025-12-16T03:01:09.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30882025-12-16T03:01:09.584ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30892025-12-16T03:01:09.588ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
30902025-12-16T03:01:09.588ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30912025-12-16T03:01:09.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30922025-12-16T03:01:09.591ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30932025-12-16T03:01:09.594ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
30942025-12-16T03:01:09.594ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30952025-12-16T03:01:09.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30962025-12-16T03:01:09.597ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30972025-12-16T03:01:09.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30982025-12-16T03:01:09.598ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
30992025-12-16T03:01:09.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
31002025-12-16T03:01:09.602ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31012025-12-16T03:01:09.602ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31022025-12-16T03:01:09.606ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
31032025-12-16T03:01:09.606ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31042025-12-16T03:01:09.606ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 438.584617ms iteration = 5 status = {}
31052025-12-16T03:01:09.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
31062025-12-16T03:01:09.606ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
31072025-12-16T03:01:09.606ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
31082025-12-16T03:01:09.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
31092025-12-16T03:01:09.608ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31102025-12-16T03:01:09.610ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
31112025-12-16T03:01:09.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
31122025-12-16T03:01:09.611ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
31132025-12-16T03:01:09.611ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
31142025-12-16T03:01:09.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
31152025-12-16T03:01:09.612ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31162025-12-16T03:01:09.617ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
31172025-12-16T03:01:09.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
31182025-12-16T03:01:09.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
31192025-12-16T03:01:09.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
31202025-12-16T03:01:09.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
31212025-12-16T03:01:09.621ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31222025-12-16T03:01:09.627ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
31232025-12-16T03:01:09.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
31242025-12-16T03:01:09.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
31252025-12-16T03:01:09.632ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
31262025-12-16T03:01:09.632ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "b8dbf809-c550-4c0c-bedb-d03345ddbd08", "date": "Tue, 16 Dec 2025 03:01:09 GMT"} })
31272025-12-16T03:01:09.632ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
31282025-12-16T03:01:09.634ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
31292025-12-16T03:01:09.634ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 406.46248ms iteration = 3 status = {"errors":[],"probes_by_sled":{"b1c76b61-acee-4be6-b582-973c938d6b6e":0}}
31302025-12-16T03:01:09.647ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
31312025-12-16T03:01:09.647ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
31322025-12-16T03:01:09.647ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "42819508-e823-45f9-8c08-97e777393815", "date": "Tue, 16 Dec 2025 03:01:09 GMT"} })
31332025-12-16T03:01:09.647ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31342025-12-16T03:01:09.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31352025-12-16T03:01:09.648ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
31362025-12-16T03:01:09.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
31372025-12-16T03:01:09.649ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31382025-12-16T03:01:09.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31392025-12-16T03:01:09.651ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
31402025-12-16T03:01:09.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
31412025-12-16T03:01:09.652ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31422025-12-16T03:01:09.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31432025-12-16T03:01:09.653ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31442025-12-16T03:01:09.656ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
31452025-12-16T03:01:09.657ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
31462025-12-16T03:01:09.657ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
31472025-12-16T03:01:09.657ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
31482025-12-16T03:01:09.657ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
31492025-12-16T03:01:09.657ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "5977c885-77da-4734-ad4d-af2bedb60833", "date": "Tue, 16 Dec 2025 03:01:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
31502025-12-16T03:01:09.657ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
31512025-12-16T03:01:09.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
31522025-12-16T03:01:09.658ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "8066b337-b381-4796-9b3d-7b40e8893a8e", "date": "Tue, 16 Dec 2025 03:01:09 GMT"} })
31532025-12-16T03:01:09.659ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "a46320af-891d-45a2-86f9-05c95076d9d6", "date": "Tue, 16 Dec 2025 03:01:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
31542025-12-16T03:01:09.659ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
31552025-12-16T03:01:09.660ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 451.010134ms iteration = 7 status = {}
31562025-12-16T03:01:09.660ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 8 reason = Dependency
31572025-12-16T03:01:09.660ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
31582025-12-16T03:01:09.660ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31592025-12-16T03:01:09.660ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1077, tv_nsec: 277763259 } })
31602025-12-16T03:01:09.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31612025-12-16T03:01:09.661ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
31622025-12-16T03:01:09.661ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "8f748e00-b840-4aa7-bc17-d4e6d7e66eb4", "date": "Tue, 16 Dec 2025 03:01:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
31632025-12-16T03:01:09.661ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
31642025-12-16T03:01:09.662ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31652025-12-16T03:01:09.663ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "b1b4a368-9d0a-4d90-9e11-8080d90fb029", "date": "Tue, 16 Dec 2025 03:01:09 GMT"} }) sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
31662025-12-16T03:01:09.663ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31672025-12-16T03:01:09.664ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
31682025-12-16T03:01:09.664ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 496.443689ms iteration = 5 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
31692025-12-16T03:01:09.665ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 25248e48-0801-41d1-9c12-2383bbe16c17 step component = FirewallRules step id = Ensure
31702025-12-16T03:01:09.665ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31712025-12-16T03:01:09.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31722025-12-16T03:01:09.666ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
31732025-12-16T03:01:09.666ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31742025-12-16T03:01:09.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31752025-12-16T03:01:09.667ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31762025-12-16T03:01:09.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31772025-12-16T03:01:09.669ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
31782025-12-16T03:01:09.669ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
31792025-12-16T03:01:09.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
31802025-12-16T03:01:09.670ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31812025-12-16T03:01:09.670ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31822025-12-16T03:01:09.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31832025-12-16T03:01:09.672ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
31842025-12-16T03:01:09.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
31852025-12-16T03:01:09.673ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31862025-12-16T03:01:09.673ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
31872025-12-16T03:01:09.673ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31882025-12-16T03:01:09.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31892025-12-16T03:01:09.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31902025-12-16T03:01:09.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
31912025-12-16T03:01:09.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
31922025-12-16T03:01:09.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31932025-12-16T03:01:09.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31942025-12-16T03:01:09.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31952025-12-16T03:01:09.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31962025-12-16T03:01:09.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31972025-12-16T03:01:09.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31982025-12-16T03:01:09.681ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
31992025-12-16T03:01:09.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
32002025-12-16T03:01:09.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
32012025-12-16T03:01:09.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32022025-12-16T03:01:09.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
32032025-12-16T03:01:09.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32042025-12-16T03:01:09.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32052025-12-16T03:01:09.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32062025-12-16T03:01:09.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32072025-12-16T03:01:09.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32082025-12-16T03:01:09.687ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
32092025-12-16T03:01:09.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
32102025-12-16T03:01:09.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32112025-12-16T03:01:09.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
32122025-12-16T03:01:09.689ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32132025-12-16T03:01:09.690ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
32142025-12-16T03:01:09.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
32152025-12-16T03:01:09.690ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
32162025-12-16T03:01:09.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
32172025-12-16T03:01:09.692ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32182025-12-16T03:01:09.694ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
32192025-12-16T03:01:09.694ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32202025-12-16T03:01:09.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32212025-12-16T03:01:09.696ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32222025-12-16T03:01:09.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32232025-12-16T03:01:09.697ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
32242025-12-16T03:01:09.698ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32252025-12-16T03:01:09.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32262025-12-16T03:01:09.700ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32272025-12-16T03:01:09.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32282025-12-16T03:01:09.702ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32292025-12-16T03:01:09.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32302025-12-16T03:01:09.722ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
32312025-12-16T03:01:09.722ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 512.412202ms iteration = 3 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
32322025-12-16T03:01:09.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
32332025-12-16T03:01:09.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
32342025-12-16T03:01:09.739ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
32352025-12-16T03:01:09.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
32362025-12-16T03:01:09.748ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32372025-12-16T03:01:09.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32382025-12-16T03:01:09.750ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32392025-12-16T03:01:09.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32402025-12-16T03:01:09.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32412025-12-16T03:01:09.757ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
32422025-12-16T03:01:09.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32432025-12-16T03:01:09.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32442025-12-16T03:01:09.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32452025-12-16T03:01:09.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32462025-12-16T03:01:09.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32472025-12-16T03:01:09.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32482025-12-16T03:01:09.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32492025-12-16T03:01:09.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32502025-12-16T03:01:09.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32512025-12-16T03:01:09.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32522025-12-16T03:01:09.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32532025-12-16T03:01:09.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32542025-12-16T03:01:09.771ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32552025-12-16T03:01:09.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32562025-12-16T03:01:09.775ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32572025-12-16T03:01:09.782ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
32582025-12-16T03:01:09.782ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32592025-12-16T03:01:09.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32602025-12-16T03:01:09.784ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32612025-12-16T03:01:09.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32622025-12-16T03:01:09.786ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32632025-12-16T03:01:09.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32642025-12-16T03:01:09.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32652025-12-16T03:01:09.793ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
32662025-12-16T03:01:09.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32672025-12-16T03:01:09.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32682025-12-16T03:01:09.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32692025-12-16T03:01:09.794ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
32702025-12-16T03:01:09.795ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c617672-7179-41ad-896b-f611264b5289", "content-length": "691", "date": "Tue, 16 Dec 2025 03:01:09 GMT"} })
32712025-12-16T03:01:09.795ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
32722025-12-16T03:01:09.795ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
32732025-12-16T03:01:09.795ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
32742025-12-16T03:01:09.795ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1077, tv_nsec: 412536797 } })
32752025-12-16T03:01:09.796ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 586.646792ms iteration = 5 status = {}
32762025-12-16T03:01:09.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
32772025-12-16T03:01:09.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32782025-12-16T03:01:09.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32792025-12-16T03:01:09.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
32802025-12-16T03:01:09.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
32812025-12-16T03:01:09.801ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
32822025-12-16T03:01:09.801ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
32832025-12-16T03:01:09.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32842025-12-16T03:01:09.801ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4e9d1b0-8c83-4a25-9756-96f1c1e35189", "content-length": "372", "date": "Tue, 16 Dec 2025 03:01:09 GMT"} })
32852025-12-16T03:01:09.801ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32862025-12-16T03:01:09.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32872025-12-16T03:01:09.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32882025-12-16T03:01:09.802ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
32892025-12-16T03:01:09.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
32902025-12-16T03:01:09.806ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32912025-12-16T03:01:09.806ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32922025-12-16T03:01:09.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32932025-12-16T03:01:09.809ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32942025-12-16T03:01:09.809ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
32952025-12-16T03:01:09.809ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
32962025-12-16T03:01:09.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32972025-12-16T03:01:09.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
32982025-12-16T03:01:09.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
32992025-12-16T03:01:09.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33002025-12-16T03:01:09.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
33012025-12-16T03:01:09.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33022025-12-16T03:01:09.814ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33032025-12-16T03:01:09.814ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33042025-12-16T03:01:09.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33052025-12-16T03:01:09.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33062025-12-16T03:01:09.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33072025-12-16T03:01:09.817ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33082025-12-16T03:01:09.818ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
33092025-12-16T03:01:09.818ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
33102025-12-16T03:01:09.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
33112025-12-16T03:01:09.823ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
33122025-12-16T03:01:09.823ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33132025-12-16T03:01:09.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33142025-12-16T03:01:09.825ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
33152025-12-16T03:01:09.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
33162025-12-16T03:01:09.826ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33172025-12-16T03:01:09.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33182025-12-16T03:01:09.827ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33192025-12-16T03:01:09.827ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33202025-12-16T03:01:09.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33212025-12-16T03:01:09.829ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33222025-12-16T03:01:09.831ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
33232025-12-16T03:01:09.831ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
33242025-12-16T03:01:09.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
33252025-12-16T03:01:09.834ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
33262025-12-16T03:01:09.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33272025-12-16T03:01:09.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33282025-12-16T03:01:09.835ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33292025-12-16T03:01:09.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33302025-12-16T03:01:09.836ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
33312025-12-16T03:01:09.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
33322025-12-16T03:01:09.837ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
33332025-12-16T03:01:09.837ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33342025-12-16T03:01:09.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33352025-12-16T03:01:09.838ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33362025-12-16T03:01:09.840ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], Name("nexus"): [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
33372025-12-16T03:01:09.840ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33382025-12-16T03:01:09.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33392025-12-16T03:01:09.841ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33402025-12-16T03:01:09.842ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
33412025-12-16T03:01:09.842ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
33422025-12-16T03:01:09.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
33432025-12-16T03:01:09.844ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
33442025-12-16T03:01:09.844ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33452025-12-16T03:01:09.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33462025-12-16T03:01:09.845ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
33472025-12-16T03:01:09.845ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
33482025-12-16T03:01:09.845ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
33492025-12-16T03:01:09.850ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
33502025-12-16T03:01:09.850ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
33512025-12-16T03:01:09.850ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
33522025-12-16T03:01:09.850ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
33532025-12-16T03:01:09.850ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
33542025-12-16T03:01:09.850ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
33552025-12-16T03:01:09.850ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
33562025-12-16T03:01:09.850ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
33572025-12-16T03:01:09.851ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
33582025-12-16T03:01:09.851ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
33592025-12-16T03:01:09.851ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
33602025-12-16T03:01:09.851ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33612025-12-16T03:01:09.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33622025-12-16T03:01:09.863ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
33632025-12-16T03:01:09.863ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
33642025-12-16T03:01:09.864ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
33652025-12-16T03:01:09.864ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
33662025-12-16T03:01:09.870ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "fe66c7dd-35d9-41d5-8853-19b3426fed7a", "date": "Tue, 16 Dec 2025 03:01:09 GMT"} })
33672025-12-16T03:01:09.870ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33682025-12-16T03:01:09.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33692025-12-16T03:01:09.872ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33702025-12-16T03:01:09.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33712025-12-16T03:01:09.875ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
33722025-12-16T03:01:09.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
33732025-12-16T03:01:09.878ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
33742025-12-16T03:01:09.878ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
33752025-12-16T03:01:09.883ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
33762025-12-16T03:01:09.883ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
33772025-12-16T03:01:09.883ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = e689181a-cf56-4376-bcc2-ac9a6df60c39 parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
33782025-12-16T03:01:09.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
33792025-12-16T03:01:09.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
33802025-12-16T03:01:09.885ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "cf3a25ae-282f-4caf-a43a-fb852088b07e", "date": "Tue, 16 Dec 2025 03:01:09 GMT"} })
33812025-12-16T03:01:09.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33822025-12-16T03:01:09.886ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 225.913703ms iteration = 8 status = {}
33832025-12-16T03:01:09.888ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
33842025-12-16T03:01:09.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
33852025-12-16T03:01:09.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
33862025-12-16T03:01:09.895ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
33872025-12-16T03:01:09.895ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
33882025-12-16T03:01:09.895ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 686.033772ms iteration = 9 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
33892025-12-16T03:01:09.993ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), time_created: 2025-12-16T02:59:15.195809Z, time_modified: 2025-12-16T02:59:15.195809Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
33902025-12-16T03:01:09.993ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
33912025-12-16T03:01:10.023ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33922025-12-16T03:01:10.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33932025-12-16T03:01:10.026ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33942025-12-16T03:01:10.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33952025-12-16T03:01:10.028ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33962025-12-16T03:01:10.031ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
33972025-12-16T03:01:10.031ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
33982025-12-16T03:01:10.034ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "c8dbcffd-54f6-45bd-bfd2-71ae40e9822f", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
33992025-12-16T03:01:10.036ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
34002025-12-16T03:01:10.036ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 25248e48-0801-41d1-9c12-2383bbe16c17 step component = Dns step id = Ensure
34012025-12-16T03:01:10.036ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
34022025-12-16T03:01:10.036ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
34032025-12-16T03:01:10.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
34042025-12-16T03:01:10.037ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
34052025-12-16T03:01:10.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
34062025-12-16T03:01:10.038ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34072025-12-16T03:01:10.042ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
34082025-12-16T03:01:10.090ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
34092025-12-16T03:01:10.090ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
34102025-12-16T03:01:10.090ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
34112025-12-16T03:01:10.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
34122025-12-16T03:01:10.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
34132025-12-16T03:01:10.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34142025-12-16T03:01:10.096ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
34152025-12-16T03:01:10.096ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
34162025-12-16T03:01:10.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
34172025-12-16T03:01:10.098ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
34182025-12-16T03:01:10.098ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
34192025-12-16T03:01:10.098ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
34202025-12-16T03:01:10.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
34212025-12-16T03:01:10.100ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34222025-12-16T03:01:10.104ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
34232025-12-16T03:01:10.104ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
34242025-12-16T03:01:10.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
34252025-12-16T03:01:10.107ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
34262025-12-16T03:01:10.107ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
34272025-12-16T03:01:10.107ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
34282025-12-16T03:01:10.107ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
34292025-12-16T03:01:10.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
34302025-12-16T03:01:10.108ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
34312025-12-16T03:01:10.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
34322025-12-16T03:01:10.109ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34332025-12-16T03:01:10.113ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
34342025-12-16T03:01:10.114ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
34352025-12-16T03:01:10.114ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
34362025-12-16T03:01:10.114ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
34372025-12-16T03:01:10.114ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
34382025-12-16T03:01:10.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
34392025-12-16T03:01:10.115ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34402025-12-16T03:01:10.119ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
34412025-12-16T03:01:10.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
34422025-12-16T03:01:10.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
34432025-12-16T03:01:10.121ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
34442025-12-16T03:01:10.121ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
34452025-12-16T03:01:10.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
34462025-12-16T03:01:10.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
34472025-12-16T03:01:10.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34482025-12-16T03:01:10.126ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
34492025-12-16T03:01:10.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
34502025-12-16T03:01:10.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
34512025-12-16T03:01:10.128ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
34522025-12-16T03:01:10.128ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
34532025-12-16T03:01:10.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
34542025-12-16T03:01:10.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
34552025-12-16T03:01:10.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34562025-12-16T03:01:10.132ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
34572025-12-16T03:01:10.132ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
34582025-12-16T03:01:10.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
34592025-12-16T03:01:10.134ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
34602025-12-16T03:01:10.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
34612025-12-16T03:01:10.134ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
34622025-12-16T03:01:10.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
34632025-12-16T03:01:10.136ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34642025-12-16T03:01:10.140ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
34652025-12-16T03:01:10.142ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
34662025-12-16T03:01:10.142ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
34672025-12-16T03:01:10.142ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 25248e48-0801-41d1-9c12-2383bbe16c17 step component = OmicronZones step id = Cleanup
34682025-12-16T03:01:10.142ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 25248e48-0801-41d1-9c12-2383bbe16c17 step component = OmicronZones step id = Cleanup
34692025-12-16T03:01:10.142ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 25248e48-0801-41d1-9c12-2383bbe16c17 step component = PhysicalDisks step id = Cleanup
34702025-12-16T03:01:10.142ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 25248e48-0801-41d1-9c12-2383bbe16c17 step component = Clickhouse step id = Ensure
34712025-12-16T03:01:10.142ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 25248e48-0801-41d1-9c12-2383bbe16c17 step component = Clickhouse step id = Ensure
34722025-12-16T03:01:10.172ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
34732025-12-16T03:01:10.177ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "65b218f6-1d8d-4c72-88bc-73aeb6baedf3", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
34742025-12-16T03:01:10.178ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "e3c0ab2b-6a1a-4146-9d34-ab9be1c43673", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
34752025-12-16T03:01:10.178ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
34762025-12-16T03:01:10.178ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
34772025-12-16T03:01:10.178ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6fc0ed0-3bae-4f67-88ff-65a2dd256d24", "content-length": "682", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
34782025-12-16T03:01:10.178ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 25248e48-0801-41d1-9c12-2383bbe16c17 step component = SupportBundles step id = Cleanup
34792025-12-16T03:01:10.178ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34802025-12-16T03:01:10.178ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
34812025-12-16T03:01:10.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
34822025-12-16T03:01:10.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34832025-12-16T03:01:10.182ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34842025-12-16T03:01:10.183ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
34852025-12-16T03:01:10.183ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
34862025-12-16T03:01:10.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
34872025-12-16T03:01:10.185ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
34882025-12-16T03:01:10.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
34892025-12-16T03:01:10.186ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34902025-12-16T03:01:10.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34912025-12-16T03:01:10.187ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34922025-12-16T03:01:10.190ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34932025-12-16T03:01:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34942025-12-16T03:01:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
34952025-12-16T03:01:10.192ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
34962025-12-16T03:01:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
34972025-12-16T03:01:10.199ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 990.075394ms iteration = 5 status = {}
34982025-12-16T03:01:10.242ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 25248e48-0801-41d1-9c12-2383bbe16c17 step component = OmicronZones step id = Cleanup
34992025-12-16T03:01:10.242ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
35002025-12-16T03:01:10.242ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
35012025-12-16T03:01:10.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
35022025-12-16T03:01:10.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
35032025-12-16T03:01:10.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35042025-12-16T03:01:10.245ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
35052025-12-16T03:01:10.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
35062025-12-16T03:01:10.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
35072025-12-16T03:01:10.291ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
35082025-12-16T03:01:10.291ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(a91734db-9ac7-4793-9c32-ddbb21f8d986 (blueprint), false)
35092025-12-16T03:01:10.291ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 25248e48-0801-41d1-9c12-2383bbe16c17 step component = Cockroach step id = Ensure
35102025-12-16T03:01:10.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
35112025-12-16T03:01:10.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
35122025-12-16T03:01:10.318ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
35132025-12-16T03:01:10.318ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 25248e48-0801-41d1-9c12-2383bbe16c17 step component = MgsUpdates step id = Ensure
35142025-12-16T03:01:10.318ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 8 reason = Dependency
35152025-12-16T03:01:10.318ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 4 reason = Dependency
35162025-12-16T03:01:10.318ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
35172025-12-16T03:01:10.318ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
35182025-12-16T03:01:10.318ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.15074023s iteration = 4 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"3479968e-502f-42b6-bbcd-88708e0aab3d","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"3479968e-502f-42b6-bbcd-88708e0aab3d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":4400,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":89902449,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":89902399,"secs":0}},"event_index":1,"execution_id":"3479968e-502f-42b6-bbcd-88708e0aab3d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":89918669,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":103706734,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":103706694,"secs":0}},"event_index":2,"execution_id":"3479968e-502f-42b6-bbcd-88708e0aab3d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":193718034,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":128743511,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":128743471,"secs":0}},"event_index":3,"execution_id":"3479968e-502f-42b6-bbcd-88708e0aab3d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":322493745,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":174920876,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":174920836,"secs":0}},"event_index":4,"execution_id":"3479968e-502f-42b6-bbcd-88708e0aab3d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":497451252,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":370823218,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":370823178,"secs":0}},"event_index":5,"execution_id":"3479968e-502f-42b6-bbcd-88708e0aab3d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":868302391,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":105851001,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":105850951,"secs":0}},"event_index":6,"execution_id":"3479968e-502f-42b6-bbcd-88708e0aab3d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":974167603,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4590,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4550,"secs":0}},"event_index":7,"execution_id":"3479968e-502f-42b6-bbcd-88708e0aab3d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":974184573,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2190,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2150,"secs":0}},"event_index":8,"execution_id":"3479968e-502f-42b6-bbcd-88708e0aab3d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":974188943,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3050,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3010,"secs":0}},"event_index":9,"execution_id":"3479968e-502f-42b6-bbcd-88708e0aab3d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":974194343,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1820,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1780,"secs":0}},"event_index":10,"execution_id":"3479968e-502f-42b6-bbcd-88708e0aab3d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":974197783,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":36645824,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":36645784,"secs":0}},"event_index":11,"execution_id":"3479968e-502f-42b6-bbcd-88708e0aab3d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":10845427,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":63209700,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":63209670,"secs":0}},"event_index":12,"execution_id":"3479968e-502f-42b6-bbcd-88708e0aab3d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":74064558,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":49002281,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":49002241,"secs":0}},"event_index":13,"execution_id":"3479968e-502f-42b6-bbcd-88708e0aab3d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":123085160,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":27302176,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":27302136,"secs":0}},"event_index":14,"execution_id":"3479968e-502f-42b6-bbcd-88708e0aab3d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":150396786,"secs":1}},{"data":{"attempt_elapsed":{"nanos":8560,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":8520,"secs":0}},"event_index":15,"execution_id":"3479968e-502f-42b6-bbcd-88708e0aab3d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":150415876,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986"}
35192025-12-16T03:01:10.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
35202025-12-16T03:01:10.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
35212025-12-16T03:01:10.320ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35222025-12-16T03:01:10.320ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35232025-12-16T03:01:10.323ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
35242025-12-16T03:01:10.323ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
35252025-12-16T03:01:10.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
35262025-12-16T03:01:10.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
35272025-12-16T03:01:10.324ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
35282025-12-16T03:01:10.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
35292025-12-16T03:01:10.367ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
35302025-12-16T03:01:10.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
35312025-12-16T03:01:10.369ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35322025-12-16T03:01:10.371ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
35332025-12-16T03:01:10.371ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
35342025-12-16T03:01:10.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
35352025-12-16T03:01:10.415ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
35362025-12-16T03:01:10.415ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 97.189761ms iteration = 8 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
35372025-12-16T03:01:10.431ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(2ff2c680-3e77-43d1-87e0-51edcb3fda30, 0), (1e5906a3-cfae-4815-8c76-8082f2ae9f87, 0), (663629f5-b2a5-4998-9d76-28f70bfaf87e, 0), (993293d3-f4d1-4544-a5c6-b0c29460233f, 0)] collection_id = 2ff2c680-3e77-43d1-87e0-51edcb3fda30
35382025-12-16T03:01:10.431ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
35392025-12-16T03:01:10.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
35402025-12-16T03:01:10.613ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 2ff2c680-3e77-43d1-87e0-51edcb3fda30 file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
35412025-12-16T03:01:10.613ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
35422025-12-16T03:01:10.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
35432025-12-16T03:01:10.615ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1e5906a3-cfae-4815-8c76-8082f2ae9f87, 0), (663629f5-b2a5-4998-9d76-28f70bfaf87e, 0), (993293d3-f4d1-4544-a5c6-b0c29460233f, 0)]
35442025-12-16T03:01:10.616ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1078, tv_nsec: 233163794 } })
35452025-12-16T03:01:10.650ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("20488683-5778-4090-b36e-e6f0d80a682b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("35dbaad1-8536-44a0-8a0a-35d7c39dcfbb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b57fa226-2428-4a54-b85d-60409e28b484.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("d812a3e2-48ae-4812-a1cc-7eb7623e4e00.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::5)) }], valid_until: Instant { tv_sec: 1078, tv_nsec: 267465688 } })
35462025-12-16T03:01:10.651ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
35472025-12-16T03:01:10.651ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
35482025-12-16T03:01:10.651ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
35492025-12-16T03:01:10.652ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fb577b4-9278-4110-aaa5-ba20bc95455e", "content-length": "476", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35502025-12-16T03:01:10.652ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
35512025-12-16T03:01:10.652ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf2b4d1f-7014-490b-aabf-c512d3bad7b7", "content-length": "734", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35522025-12-16T03:01:10.652ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
35532025-12-16T03:01:10.652ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "58e3b380-51de-4a0e-9e95-4dd567e9f52c", "content-length": "111", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35542025-12-16T03:01:10.652ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
35552025-12-16T03:01:10.653ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bcf79fb5-647a-41db-8e85-5e45726b261f", "content-length": "111", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35562025-12-16T03:01:10.653ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
35572025-12-16T03:01:10.653ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b472891-3e6b-4844-bffe-9e4d47f54fe8", "content-length": "163", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35582025-12-16T03:01:10.653ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
35592025-12-16T03:01:10.654ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f61d6a7-feb4-4e6c-9c2b-fba3e86c650e", "content-length": "163", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35602025-12-16T03:01:10.654ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
35612025-12-16T03:01:10.654ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "64fbcca7-b20a-49f2-a863-7adf605eb38e", "content-length": "166", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35622025-12-16T03:01:10.654ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
35632025-12-16T03:01:10.654ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "afc7a5c9-01c0-4527-8240-01754c79dc76", "content-length": "166", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35642025-12-16T03:01:10.654ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
35652025-12-16T03:01:10.655ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "06761566-c6e6-447d-b426-1200eb191634", "content-length": "702", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35662025-12-16T03:01:10.655ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
35672025-12-16T03:01:10.656ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c5ebaa4-eacb-491a-b84a-ffb8e6761bd2", "content-length": "727", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35682025-12-16T03:01:10.656ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
35692025-12-16T03:01:10.656ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4fb45b90-37da-4637-85dd-0307e13ea503", "content-length": "729", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35702025-12-16T03:01:10.656ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
35712025-12-16T03:01:10.656ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d941b357-ce09-49ae-a499-b84dcda487c1", "content-length": "728", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35722025-12-16T03:01:10.656ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
35732025-12-16T03:01:10.656ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "02580aa6-a4c4-4a24-ae6e-a0c809ba4d1e", "content-length": "722", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35742025-12-16T03:01:10.656ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
35752025-12-16T03:01:10.657ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "6425a74f-ec85-4c4e-9e3c-d1b8523ec7d6", "content-length": "10", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35762025-12-16T03:01:10.657ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
35772025-12-16T03:01:10.657ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c170394-c5cc-4e1b-99a7-0aabc814c537", "content-length": "144", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35782025-12-16T03:01:10.657ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
35792025-12-16T03:01:10.657ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "06628207-1fb4-464b-9191-3a8f8454ec5e", "content-length": "147", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35802025-12-16T03:01:10.657ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
35812025-12-16T03:01:10.658ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c93ce20-774a-44cb-88fe-2932cb1e9a3f", "content-length": "109", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35822025-12-16T03:01:10.658ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
35832025-12-16T03:01:10.658ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b4bc96e-84bb-413b-ba47-83cf24850fbf", "content-length": "109", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35842025-12-16T03:01:10.658ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
35852025-12-16T03:01:10.658ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6c2a9a0-9c19-4380-903d-14116c59c48e", "content-length": "163", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35862025-12-16T03:01:10.659ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
35872025-12-16T03:01:10.659ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "949ca853-43da-4182-acf4-f589e7b0ef7a", "content-length": "163", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35882025-12-16T03:01:10.659ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
35892025-12-16T03:01:10.659ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0fa7cd52-e055-4767-a6c6-3afa814aae45", "content-length": "166", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35902025-12-16T03:01:10.659ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
35912025-12-16T03:01:10.660ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "76adc494-1ecc-4c0b-94e9-5483cab4bb74", "content-length": "166", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35922025-12-16T03:01:10.660ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
35932025-12-16T03:01:10.660ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a5e6452-4751-4ba3-a9e5-6505a2a0e237", "content-length": "702", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35942025-12-16T03:01:10.660ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
35952025-12-16T03:01:10.660ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "50fc4f88-01dc-413c-949e-1b3152800272", "content-length": "727", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35962025-12-16T03:01:10.660ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
35972025-12-16T03:01:10.660ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d88ffa8-d16b-4148-ac12-8b0ee3842d44", "content-length": "729", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
35982025-12-16T03:01:10.660ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
35992025-12-16T03:01:10.661ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7c50e0e-75ed-480a-9558-bf94c8ebe4ce", "content-length": "728", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
36002025-12-16T03:01:10.661ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
36012025-12-16T03:01:10.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
36022025-12-16T03:01:10.663ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36032025-12-16T03:01:10.665ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
36042025-12-16T03:01:10.665ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
36052025-12-16T03:01:10.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
36062025-12-16T03:01:10.667ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36072025-12-16T03:01:10.669ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
36082025-12-16T03:01:10.669ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
36092025-12-16T03:01:10.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
36102025-12-16T03:01:10.703ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
36112025-12-16T03:01:10.703ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
36122025-12-16T03:01:10.750ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a5c90e3-9196-4099-8b99-44df370f736a", "content-length": "95610", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
36132025-12-16T03:01:10.753ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
36142025-12-16T03:01:10.753ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
36152025-12-16T03:01:10.753ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
36162025-12-16T03:01:10.753ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
36172025-12-16T03:01:10.753ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
36182025-12-16T03:01:10.753ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
36192025-12-16T03:01:10.753ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
36202025-12-16T03:01:10.753ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
36212025-12-16T03:01:10.753ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "57e726eb-a1d5-4ec0-aac3-da4d8998d0b4", "content-length": "64", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
36222025-12-16T03:01:10.753ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "47e04317-0969-4bee-b989-15d336fa6254", "content-length": "64", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
36232025-12-16T03:01:10.753ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
36242025-12-16T03:01:10.753ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
36252025-12-16T03:01:10.753ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5b015cf-368f-441b-a58a-655f60cb793b", "content-length": "64", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
36262025-12-16T03:01:10.753ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "01409705-33e1-4f56-b63c-1adde7b44134", "content-length": "64", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
36272025-12-16T03:01:10.753ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
36282025-12-16T03:01:10.753ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
36292025-12-16T03:01:10.754ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1d938b1-7711-492a-b3bc-c68d307ec246", "content-length": "64", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
36302025-12-16T03:01:10.754ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
36312025-12-16T03:01:10.791ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf4d3a6c-94e0-4305-8ed8-b6bb7ae473f8", "content-length": "388712", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
36322025-12-16T03:01:10.792ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "94c7ecc7-1262-4b93-8061-4ec53a10f63c", "content-length": "384311", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
36332025-12-16T03:01:10.793ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9b935e9-6eef-4366-b6d1-ac2a31cc3e37", "content-length": "385417", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
36342025-12-16T03:01:10.795ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3111c8e-04bb-4874-96ad-2484d9d1ed73", "content-length": "418617", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
36352025-12-16T03:01:10.797ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "baaff569-d010-452a-b5b4-1e51058735d3", "content-length": "384664", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
36362025-12-16T03:01:10.800ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
36372025-12-16T03:01:10.800ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
36382025-12-16T03:01:10.800ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
36392025-12-16T03:01:10.800ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
36402025-12-16T03:01:10.800ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
36412025-12-16T03:01:10.839ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c (omicron_zone)
36422025-12-16T03:01:10.839ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
36432025-12-16T03:01:10.842ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "47aae1ec-152f-49af-a17b-99bec6d96908", "content-length": "125", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
36442025-12-16T03:01:10.844ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
36452025-12-16T03:01:10.941ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
36462025-12-16T03:01:10.941ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
36472025-12-16T03:01:10.942ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1396193c-c516-49a4-a09b-e3b38103bc7e", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
36482025-12-16T03:01:10.942ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
36492025-12-16T03:01:10.943ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
36502025-12-16T03:01:10.943ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
36512025-12-16T03:01:10.943ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "4567126e-c976-4f7b-b949-f9d2e881ef29", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
36522025-12-16T03:01:10.943ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
36532025-12-16T03:01:10.944ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
36542025-12-16T03:01:10.944ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
36552025-12-16T03:01:10.945ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe397d2a-1bcb-4b9a-82d9-d19781763ba1", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:01:10 GMT"} })
36562025-12-16T03:01:10.945ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
36572025-12-16T03:01:10.945ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
36582025-12-16T03:01:10.945ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
36592025-12-16T03:01:10.947ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
36602025-12-16T03:01:10.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
36612025-12-16T03:01:10.949ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36622025-12-16T03:01:10.952ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
36632025-12-16T03:01:10.952ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
36642025-12-16T03:01:10.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
36652025-12-16T03:01:11.060ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 284 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 4f1bf91b-ba92-4eeb-9067-7ce9573445a6 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
36662025-12-16T03:01:11.144ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = cd9f1d61-49c8-4ef5-9e52-7e084cb4a60e file = nexus/db-queries/src/db/datastore/inventory.rs:1613
36672025-12-16T03:01:11.144ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = cd9f1d61-49c8-4ef5-9e52-7e084cb4a60e time_started = 2025-12-16 03:01:10.651393 UTC
36682025-12-16T03:01:11.144ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 11 reason = Dependency
36692025-12-16T03:01:11.144ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
36702025-12-16T03:01:11.144ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 13 reason = Dependency
36712025-12-16T03:01:11.144ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 18.25\u{b5}s iteration = 11 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
36722025-12-16T03:01:11.144ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
36732025-12-16T03:01:11.144ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 825.6222ms iteration = 4 status = {"collection_id":"cd9f1d61-49c8-4ef5-9e52-7e084cb4a60e","time_done":"2025-12-16 03:01:10.945934 UTC","time_started":"2025-12-16 03:01:10.651393 UTC"}
36742025-12-16T03:01:11.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
36752025-12-16T03:01:11.146ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36762025-12-16T03:01:11.149ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
36772025-12-16T03:01:11.149ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
36782025-12-16T03:01:11.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
36792025-12-16T03:01:11.236ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
36802025-12-16T03:01:11.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
36812025-12-16T03:01:11.298ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 153.94071ms iteration = 13 status = {"Loaded":{"collection_id":"fecebf81-1ff4-4296-8099-25ffd5550e29","time_loaded":"2025-12-16T03:01:11.144167462Z","time_started":"2025-12-16T03:01:10.658135Z"}}
36822025-12-16T03:01:11.298ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 12 reason = Dependency
36832025-12-16T03:01:11.298ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 6 reason = Dependency
36842025-12-16T03:01:11.298ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
36852025-12-16T03:01:11.298ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 10 reason = Dependency
36862025-12-16T03:01:11.298ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 10 reason = Dependency
36872025-12-16T03:01:11.298ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 22.93\u{b5}s iteration = 12 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
36882025-12-16T03:01:11.298ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
36892025-12-16T03:01:11.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36902025-12-16T03:01:11.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
36912025-12-16T03:01:11.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
36922025-12-16T03:01:11.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
36932025-12-16T03:01:11.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36942025-12-16T03:01:11.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36952025-12-16T03:01:11.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36962025-12-16T03:01:11.300ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36972025-12-16T03:01:11.303ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
36982025-12-16T03:01:11.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36992025-12-16T03:01:11.303ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
37002025-12-16T03:01:11.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
37012025-12-16T03:01:11.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37022025-12-16T03:01:11.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
37032025-12-16T03:01:11.303ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
37042025-12-16T03:01:11.303ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37052025-12-16T03:01:11.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37062025-12-16T03:01:11.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37072025-12-16T03:01:11.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37082025-12-16T03:01:11.306ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37092025-12-16T03:01:11.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37102025-12-16T03:01:11.307ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
37112025-12-16T03:01:11.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
37122025-12-16T03:01:11.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
37132025-12-16T03:01:11.307ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
37142025-12-16T03:01:11.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37152025-12-16T03:01:11.307ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37162025-12-16T03:01:11.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37172025-12-16T03:01:11.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
37182025-12-16T03:01:11.310ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
37192025-12-16T03:01:11.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
37202025-12-16T03:01:11.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
37212025-12-16T03:01:11.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
37222025-12-16T03:01:11.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37232025-12-16T03:01:11.316ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
37242025-12-16T03:01:11.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
37252025-12-16T03:01:11.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
37262025-12-16T03:01:11.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37272025-12-16T03:01:11.322ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
37282025-12-16T03:01:11.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
37292025-12-16T03:01:11.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
37302025-12-16T03:01:11.325ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
37312025-12-16T03:01:11.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
37322025-12-16T03:01:11.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
37332025-12-16T03:01:11.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37342025-12-16T03:01:11.329ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
37352025-12-16T03:01:11.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
37362025-12-16T03:01:11.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
37372025-12-16T03:01:11.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37382025-12-16T03:01:11.333ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
37392025-12-16T03:01:11.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
37402025-12-16T03:01:11.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
37412025-12-16T03:01:11.336ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
37422025-12-16T03:01:11.336ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 38.63425ms iteration = 6 status = {"blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","inventory_collection_id":"fecebf81-1ff4-4296-8099-25ffd5550e29","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
37432025-12-16T03:01:11.368ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 70.25835ms iteration = 10 status = {"physical_disks_added":0}
37442025-12-16T03:01:11.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37452025-12-16T03:01:11.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37462025-12-16T03:01:11.404ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37472025-12-16T03:01:11.406ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
37482025-12-16T03:01:11.406ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37492025-12-16T03:01:11.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37502025-12-16T03:01:11.408ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37512025-12-16T03:01:11.410ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
37522025-12-16T03:01:11.410ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37532025-12-16T03:01:11.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37542025-12-16T03:01:11.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37552025-12-16T03:01:11.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37562025-12-16T03:01:11.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37572025-12-16T03:01:11.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37582025-12-16T03:01:11.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37592025-12-16T03:01:11.456ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
37602025-12-16T03:01:11.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37612025-12-16T03:01:11.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37622025-12-16T03:01:11.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37632025-12-16T03:01:11.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37642025-12-16T03:01:11.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37652025-12-16T03:01:11.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
37662025-12-16T03:01:11.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37672025-12-16T03:01:11.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37682025-12-16T03:01:11.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37692025-12-16T03:01:11.467ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
37702025-12-16T03:01:11.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37712025-12-16T03:01:11.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37722025-12-16T03:01:11.469ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37732025-12-16T03:01:11.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37742025-12-16T03:01:11.470ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37752025-12-16T03:01:11.473ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
37762025-12-16T03:01:11.473ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37772025-12-16T03:01:11.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37782025-12-16T03:01:11.475ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37792025-12-16T03:01:11.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37802025-12-16T03:01:11.476ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37812025-12-16T03:01:11.480ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
37822025-12-16T03:01:11.480ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37832025-12-16T03:01:11.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37842025-12-16T03:01:11.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37852025-12-16T03:01:11.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37862025-12-16T03:01:11.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37872025-12-16T03:01:11.485ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
37882025-12-16T03:01:11.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37892025-12-16T03:01:11.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37902025-12-16T03:01:11.498ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37912025-12-16T03:01:11.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37922025-12-16T03:01:11.500ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37932025-12-16T03:01:11.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37942025-12-16T03:01:11.502ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37952025-12-16T03:01:11.505ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
37962025-12-16T03:01:11.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37972025-12-16T03:01:11.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37982025-12-16T03:01:11.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37992025-12-16T03:01:11.509ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
38002025-12-16T03:01:11.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38012025-12-16T03:01:11.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38022025-12-16T03:01:11.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38032025-12-16T03:01:11.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38042025-12-16T03:01:11.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38052025-12-16T03:01:11.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38062025-12-16T03:01:11.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38072025-12-16T03:01:11.519ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
38082025-12-16T03:01:11.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38092025-12-16T03:01:11.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38102025-12-16T03:01:11.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38112025-12-16T03:01:11.523ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
38122025-12-16T03:01:11.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38132025-12-16T03:01:11.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38142025-12-16T03:01:11.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38152025-12-16T03:01:11.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38162025-12-16T03:01:11.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38172025-12-16T03:01:11.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38182025-12-16T03:01:11.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38192025-12-16T03:01:11.532ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
38202025-12-16T03:01:11.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38212025-12-16T03:01:11.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38222025-12-16T03:01:11.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38232025-12-16T03:01:11.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38242025-12-16T03:01:11.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38252025-12-16T03:01:11.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
38262025-12-16T03:01:11.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38272025-12-16T03:01:11.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38282025-12-16T03:01:11.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38292025-12-16T03:01:11.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38302025-12-16T03:01:11.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38312025-12-16T03:01:11.544ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
38322025-12-16T03:01:11.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38332025-12-16T03:01:11.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38342025-12-16T03:01:11.546ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38352025-12-16T03:01:11.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38362025-12-16T03:01:11.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38372025-12-16T03:01:11.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
38382025-12-16T03:01:11.551ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38392025-12-16T03:01:11.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38402025-12-16T03:01:11.553ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38412025-12-16T03:01:11.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38422025-12-16T03:01:11.554ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38432025-12-16T03:01:11.557ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
38442025-12-16T03:01:11.557ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38452025-12-16T03:01:11.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38462025-12-16T03:01:11.559ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38472025-12-16T03:01:11.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38482025-12-16T03:01:11.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38492025-12-16T03:01:11.562ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
38502025-12-16T03:01:11.562ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38512025-12-16T03:01:11.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38522025-12-16T03:01:11.604ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
38532025-12-16T03:01:11.604ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
38542025-12-16T03:01:11.604ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 465d6f86-5b45-4ad4-b48d-6eac207aa8b2 parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
38552025-12-16T03:01:11.604ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38562025-12-16T03:01:11.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38572025-12-16T03:01:11.606ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38582025-12-16T03:01:11.609ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
38592025-12-16T03:01:11.609ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
38602025-12-16T03:01:11.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
38612025-12-16T03:01:11.611ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
38622025-12-16T03:01:11.611ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
38632025-12-16T03:01:11.611ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 313.811315ms iteration = 10 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
38642025-12-16T03:01:14.208ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 26 reason = Timeout
38652025-12-16T03:01:14.209ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
38662025-12-16T03:01:14.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
38672025-12-16T03:01:14.210ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38682025-12-16T03:01:14.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
38692025-12-16T03:01:14.214ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
38702025-12-16T03:01:14.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
38712025-12-16T03:01:14.216ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
38722025-12-16T03:01:14.216ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.732238ms iteration = 26 status = {"config_updated":false}
38732025-12-16T03:01:19.208ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 14 reason = Timeout
38742025-12-16T03:01:19.208ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 27 reason = Timeout
38752025-12-16T03:01:19.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38762025-12-16T03:01:19.208ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
38772025-12-16T03:01:19.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
38782025-12-16T03:01:19.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
38792025-12-16T03:01:19.210ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 14 reason = Timeout
38802025-12-16T03:01:19.210ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
38812025-12-16T03:01:19.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38822025-12-16T03:01:19.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38832025-12-16T03:01:19.213ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38842025-12-16T03:01:19.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
38852025-12-16T03:01:19.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38862025-12-16T03:01:19.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
38872025-12-16T03:01:19.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38882025-12-16T03:01:19.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
38892025-12-16T03:01:19.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
38902025-12-16T03:01:19.216ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
38912025-12-16T03:01:19.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
38922025-12-16T03:01:19.218ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
38932025-12-16T03:01:19.218ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 9.315327ms iteration = 27 status = {"config_updated":false}
38942025-12-16T03:01:19.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38952025-12-16T03:01:19.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
38962025-12-16T03:01:19.221ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38972025-12-16T03:01:19.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
38982025-12-16T03:01:19.237ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
38992025-12-16T03:01:19.237ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.067239ms iteration = 14 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
39002025-12-16T03:01:19.248ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39012025-12-16T03:01:19.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39022025-12-16T03:01:19.249ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 39.346079ms iteration = 14 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
39032025-12-16T03:01:21.060ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 327 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 49dfe4b0-f835-4b3a-b2bf-09cd669d8d15 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
39042025-12-16T03:01:24.209ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 14 reason = Timeout
39052025-12-16T03:01:24.209ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 28 reason = Timeout
39062025-12-16T03:01:24.209ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
39072025-12-16T03:01:24.209ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
39082025-12-16T03:01:24.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
39092025-12-16T03:01:24.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
39102025-12-16T03:01:24.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39112025-12-16T03:01:24.212ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39122025-12-16T03:01:24.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
39132025-12-16T03:01:24.216ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
39142025-12-16T03:01:24.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
39152025-12-16T03:01:24.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
39162025-12-16T03:01:24.216ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
39172025-12-16T03:01:24.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
39182025-12-16T03:01:24.217ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
39192025-12-16T03:01:24.217ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.759678ms iteration = 28 status = {"config_updated":false}
39202025-12-16T03:01:24.218ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = fecebf81-1ff4-4296-8099-25ffd5550e29 original_time_started = 2025-12-16 03:01:10.658135 UTC
39212025-12-16T03:01:24.218ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.973984ms iteration = 14 status = {"Loaded":{"collection_id":"fecebf81-1ff4-4296-8099-25ffd5550e29","time_loaded":"2025-12-16T03:01:24.209605610Z","time_started":"2025-12-16T03:01:10.658135Z"}}
39222025-12-16T03:01:24.227ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 10 reason = Timeout
39232025-12-16T03:01:24.227ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 10 reason = Timeout
39242025-12-16T03:01:24.227ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
39252025-12-16T03:01:24.227ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
39262025-12-16T03:01:24.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
39272025-12-16T03:01:24.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
39282025-12-16T03:01:24.228ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
39292025-12-16T03:01:24.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
39302025-12-16T03:01:24.228ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39312025-12-16T03:01:24.229ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39322025-12-16T03:01:24.230ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
39332025-12-16T03:01:24.231ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
39342025-12-16T03:01:24.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
39352025-12-16T03:01:24.231ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
39362025-12-16T03:01:24.232ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.190796ms iteration = 10 status = "NoSitrep"
39372025-12-16T03:01:24.234ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
39382025-12-16T03:01:24.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
39392025-12-16T03:01:24.234ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
39402025-12-16T03:01:24.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
39412025-12-16T03:01:24.236ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39422025-12-16T03:01:24.237ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
39432025-12-16T03:01:24.238ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
39442025-12-16T03:01:24.238ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 11.364214ms iteration = 10 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
39452025-12-16T03:01:29.209ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 15 reason = Timeout
39462025-12-16T03:01:29.209ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 29 reason = Timeout
39472025-12-16T03:01:29.209ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
39482025-12-16T03:01:29.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39492025-12-16T03:01:29.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
39502025-12-16T03:01:29.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
39512025-12-16T03:01:29.210ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 15 reason = Timeout
39522025-12-16T03:01:29.210ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39532025-12-16T03:01:29.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39542025-12-16T03:01:29.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39552025-12-16T03:01:29.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39562025-12-16T03:01:29.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
39572025-12-16T03:01:29.214ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
39582025-12-16T03:01:29.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
39592025-12-16T03:01:29.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
39602025-12-16T03:01:29.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39612025-12-16T03:01:29.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
39622025-12-16T03:01:29.216ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
39632025-12-16T03:01:29.216ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.162821ms iteration = 29 status = {"config_updated":false}
39642025-12-16T03:01:29.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39652025-12-16T03:01:29.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
39662025-12-16T03:01:29.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39672025-12-16T03:01:29.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
39682025-12-16T03:01:29.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39692025-12-16T03:01:29.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
39702025-12-16T03:01:29.233ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
39712025-12-16T03:01:29.233ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.068135ms iteration = 15 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
39722025-12-16T03:01:29.247ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39732025-12-16T03:01:29.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39742025-12-16T03:01:29.249ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 38.999734ms iteration = 15 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
39752025-12-16T03:01:31.061ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 492 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 7c610e26-7374-4b7b-8f3e-4f69c994242b response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
39762025-12-16T03:01:34.209ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 30 reason = Timeout
39772025-12-16T03:01:34.209ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
39782025-12-16T03:01:34.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
39792025-12-16T03:01:34.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39802025-12-16T03:01:34.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
39812025-12-16T03:01:34.214ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
39822025-12-16T03:01:34.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
39832025-12-16T03:01:34.216ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
39842025-12-16T03:01:34.216ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.251562ms iteration = 30 status = {"config_updated":false}
39852025-12-16T03:01:39.168ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 6 reason = Timeout
39862025-12-16T03:01:39.168ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 6 reason = Timeout
39872025-12-16T03:01:39.168ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 6 reason = Timeout
39882025-12-16T03:01:39.168ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
39892025-12-16T03:01:39.168ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
39902025-12-16T03:01:39.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
39912025-12-16T03:01:39.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
39922025-12-16T03:01:39.174ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1106, tv_nsec: 788786744 } })
39932025-12-16T03:01:39.174ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
39942025-12-16T03:01:39.174ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1106, tv_nsec: 789713836 } })
39952025-12-16T03:01:39.175ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1106, tv_nsec: 792119046 } })
39962025-12-16T03:01:39.178ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 10.586674ms iteration = 6 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
39972025-12-16T03:01:39.209ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 15 reason = Timeout
39982025-12-16T03:01:39.209ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 16 reason = Timeout
39992025-12-16T03:01:39.209ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 31 reason = Timeout
40002025-12-16T03:01:39.209ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
40012025-12-16T03:01:39.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40022025-12-16T03:01:39.209ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
40032025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
40042025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
40052025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
40062025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 6 reason = Timeout
40072025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 9 reason = Timeout
40082025-12-16T03:01:39.210ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
40092025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40102025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40112025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 11 reason = Timeout
40122025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
40132025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 6 reason = Timeout
40142025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 6 reason = Timeout
40152025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
40162025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 9 reason = Timeout
40172025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
40182025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
40192025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 6 reason = Timeout
40202025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 6 reason = Timeout
40212025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
40222025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40232025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
40242025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 16 reason = Timeout
40252025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
40262025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 6 reason = Timeout
40272025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
40282025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
40292025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40302025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40312025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
40322025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
40332025-12-16T03:01:39.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
40342025-12-16T03:01:39.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40352025-12-16T03:01:39.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
40362025-12-16T03:01:39.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
40372025-12-16T03:01:39.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
40382025-12-16T03:01:39.212ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40392025-12-16T03:01:39.212ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40402025-12-16T03:01:39.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40412025-12-16T03:01:39.212ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40422025-12-16T03:01:39.213ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40432025-12-16T03:01:39.213ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40442025-12-16T03:01:39.214ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40452025-12-16T03:01:39.214ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40462025-12-16T03:01:39.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40472025-12-16T03:01:39.216ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40482025-12-16T03:01:39.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
40492025-12-16T03:01:39.218ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
40502025-12-16T03:01:39.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
40512025-12-16T03:01:39.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
40522025-12-16T03:01:39.218ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
40532025-12-16T03:01:39.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
40542025-12-16T03:01:39.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
40552025-12-16T03:01:39.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40562025-12-16T03:01:39.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
40572025-12-16T03:01:39.220ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
40582025-12-16T03:01:39.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
40592025-12-16T03:01:39.220ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
40602025-12-16T03:01:39.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
40612025-12-16T03:01:39.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
40622025-12-16T03:01:39.220ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
40632025-12-16T03:01:39.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
40642025-12-16T03:01:39.220ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
40652025-12-16T03:01:39.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
40662025-12-16T03:01:39.220ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
40672025-12-16T03:01:39.220ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40682025-12-16T03:01:39.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
40692025-12-16T03:01:39.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40702025-12-16T03:01:39.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
40712025-12-16T03:01:39.222ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1106, tv_nsec: 838917589 } })
40722025-12-16T03:01:39.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
40732025-12-16T03:01:39.222ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
40742025-12-16T03:01:39.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
40752025-12-16T03:01:39.222ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1106, tv_nsec: 839634318 } })
40762025-12-16T03:01:39.223ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
40772025-12-16T03:01:39.223ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
40782025-12-16T03:01:39.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
40792025-12-16T03:01:39.224ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
40802025-12-16T03:01:39.224ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 14.630715ms iteration = 31 status = {"config_updated":false}
40812025-12-16T03:01:39.227ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 6 reason = Timeout
40822025-12-16T03:01:39.227ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 22.21\u{b5}s iteration = 6 status = {"disabled":true,"errors":[],"sps":[]}
40832025-12-16T03:01:39.227ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
40842025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
40852025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 17.774726ms iteration = 6 status = 0
40862025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 6 reason = Timeout
40872025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 6 reason = Timeout
40882025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
40892025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
40902025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 6 reason = Timeout
40912025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 6 reason = Timeout
40922025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
40932025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
40942025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 6 reason = Timeout
40952025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
40962025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 11 reason = Timeout
40972025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
40982025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 11 reason = Timeout
40992025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
41002025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
41012025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
41022025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = fecebf81-1ff4-4296-8099-25ffd5550e29 original_time_started = 2025-12-16 03:01:10.658135 UTC
41032025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
41042025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
41052025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 19.157963ms iteration = 15 status = {"Loaded":{"collection_id":"fecebf81-1ff4-4296-8099-25ffd5550e29","time_loaded":"2025-12-16T03:01:39.209638777Z","time_started":"2025-12-16T03:01:10.658135Z"}}
41062025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
41072025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
41082025-12-16T03:01:39.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
41092025-12-16T03:01:39.229ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
41102025-12-16T03:01:39.229ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
41112025-12-16T03:01:39.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
41122025-12-16T03:01:39.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
41132025-12-16T03:01:39.231ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41142025-12-16T03:01:39.231ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41152025-12-16T03:01:39.235ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
41162025-12-16T03:01:39.235ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
41172025-12-16T03:01:39.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
41182025-12-16T03:01:39.236ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
41192025-12-16T03:01:39.236ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
41202025-12-16T03:01:39.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
41212025-12-16T03:01:39.239ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 10.840238ms iteration = 11 status = "NoSitrep"
41222025-12-16T03:01:39.265ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
41232025-12-16T03:01:39.265ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
41242025-12-16T03:01:39.265ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f40f7daf-2f0b-4735-9274-018635d9be54", "content-length": "26", "date": "Tue, 16 Dec 2025 03:01:39 GMT"} })
41252025-12-16T03:01:39.265ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
41262025-12-16T03:01:39.267ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
41272025-12-16T03:01:39.267ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1106, tv_nsec: 884481746 } })
41282025-12-16T03:01:39.281ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41292025-12-16T03:01:39.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
41302025-12-16T03:01:39.285ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
41312025-12-16T03:01:39.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
41322025-12-16T03:01:39.286ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
41332025-12-16T03:01:39.286ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
41342025-12-16T03:01:39.286ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
41352025-12-16T03:01:39.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
41362025-12-16T03:01:39.289ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
41372025-12-16T03:01:39.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
41382025-12-16T03:01:39.291ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 81.434992ms iteration = 6 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
41392025-12-16T03:01:39.295ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
41402025-12-16T03:01:39.296ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
41412025-12-16T03:01:39.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
41422025-12-16T03:01:39.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
41432025-12-16T03:01:39.296ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41442025-12-16T03:01:39.298ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 87.795132ms iteration = 16 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
41452025-12-16T03:01:39.298ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
41462025-12-16T03:01:39.298ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
41472025-12-16T03:01:39.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
41482025-12-16T03:01:39.299ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41492025-12-16T03:01:39.300ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
41502025-12-16T03:01:39.300ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 90.299694ms iteration = 9 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
41512025-12-16T03:01:39.301ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 91.418568ms iteration = 11 status = {"physical_disks_added":0}
41522025-12-16T03:01:39.302ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
41532025-12-16T03:01:39.304ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41542025-12-16T03:01:39.307ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
41552025-12-16T03:01:39.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
41562025-12-16T03:01:39.308ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 80.52271ms iteration = 6 status = {"errors":[],"finish_invoked_ok":[]}
41572025-12-16T03:01:39.309ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
41582025-12-16T03:01:39.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
41592025-12-16T03:01:39.310ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
41602025-12-16T03:01:39.310ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41612025-12-16T03:01:39.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41622025-12-16T03:01:39.311ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
41632025-12-16T03:01:39.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
41642025-12-16T03:01:39.312ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41652025-12-16T03:01:39.312ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
41662025-12-16T03:01:39.314ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 146.378175ms iteration = 6 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
41672025-12-16T03:01:39.314ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
41682025-12-16T03:01:39.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41692025-12-16T03:01:39.315ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
41702025-12-16T03:01:39.315ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 87.401907ms iteration = 11 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
41712025-12-16T03:01:39.319ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 91.067293ms iteration = 6 status = {"errors":[],"garbage_collect_requested":[]}
41722025-12-16T03:01:39.322ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 94.258344ms iteration = 6 status = {"errors":[],"requests_created_ok":[]}
41732025-12-16T03:01:39.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
41742025-12-16T03:01:39.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41752025-12-16T03:01:39.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
41762025-12-16T03:01:39.342ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
41772025-12-16T03:01:39.342ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 133.283969ms iteration = 16 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
41782025-12-16T03:01:39.348ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
41792025-12-16T03:01:39.348ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 137.866697ms iteration = 6 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
41802025-12-16T03:01:39.349ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
41812025-12-16T03:01:39.349ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
41822025-12-16T03:01:39.349ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
41832025-12-16T03:01:39.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
41842025-12-16T03:01:39.351ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
41852025-12-16T03:01:39.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
41862025-12-16T03:01:39.363ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 135.499377ms iteration = 6 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
41872025-12-16T03:01:39.366ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
41882025-12-16T03:01:39.366ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
41892025-12-16T03:01:39.366ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "768afd2a-9f58-4182-8d59-1e77cca363b5", "content-length": "26", "date": "Tue, 16 Dec 2025 03:01:39 GMT"} })
41902025-12-16T03:01:39.366ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
41912025-12-16T03:01:39.368ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
41922025-12-16T03:01:39.373ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 205.274911ms iteration = 6 status = {}
41932025-12-16T03:01:39.400ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
41942025-12-16T03:01:39.400ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
41952025-12-16T03:01:39.400ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 190.315341ms iteration = 6 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
41962025-12-16T03:01:39.401ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 172.881581ms iteration = 6 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
41972025-12-16T03:01:39.401ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
41982025-12-16T03:01:39.401ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
41992025-12-16T03:01:39.401ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "06f7f366-5a16-408c-8c39-142a10baa86f", "content-length": "372", "date": "Tue, 16 Dec 2025 03:01:39 GMT"} })
42002025-12-16T03:01:39.401ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42012025-12-16T03:01:39.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42022025-12-16T03:01:39.416ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1107, tv_nsec: 32736385 } })
42032025-12-16T03:01:39.418ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42042025-12-16T03:01:39.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
42052025-12-16T03:01:39.420ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42062025-12-16T03:01:39.422ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
42072025-12-16T03:01:39.422ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42082025-12-16T03:01:39.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
42092025-12-16T03:01:39.424ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42102025-12-16T03:01:39.426ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
42112025-12-16T03:01:39.427ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
42122025-12-16T03:01:39.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
42132025-12-16T03:01:39.449ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1107, tv_nsec: 65818313 } })
42142025-12-16T03:01:39.457ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
42152025-12-16T03:01:39.458ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
42162025-12-16T03:01:39.458ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "0c8b7251-d831-42d0-9335-6f64c2cf7a92", "date": "Tue, 16 Dec 2025 03:01:39 GMT"} })
42172025-12-16T03:01:39.458ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42182025-12-16T03:01:39.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42192025-12-16T03:01:39.461ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42202025-12-16T03:01:39.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42212025-12-16T03:01:39.464ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
42222025-12-16T03:01:39.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
42232025-12-16T03:01:39.468ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
42242025-12-16T03:01:39.468ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
42252025-12-16T03:01:39.470ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c38e8241-d3d3-42be-a62e-3b2eb1a40f77", "date": "Tue, 16 Dec 2025 03:01:39 GMT"} })
42262025-12-16T03:01:39.473ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 262.740499ms iteration = 9 status = {}
42272025-12-16T03:01:39.481ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42282025-12-16T03:01:39.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42292025-12-16T03:01:39.482ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
42302025-12-16T03:01:39.482ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42312025-12-16T03:01:39.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42322025-12-16T03:01:39.495ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
42332025-12-16T03:01:39.495ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
42342025-12-16T03:01:39.496ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
42352025-12-16T03:01:39.496ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
42362025-12-16T03:01:39.496ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
42372025-12-16T03:01:39.496ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
42382025-12-16T03:01:39.496ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
42392025-12-16T03:01:39.496ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
42402025-12-16T03:01:39.497ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
42412025-12-16T03:01:39.497ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
42422025-12-16T03:01:39.497ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
42432025-12-16T03:01:39.497ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42442025-12-16T03:01:39.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42452025-12-16T03:01:39.499ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
42462025-12-16T03:01:39.499ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
42472025-12-16T03:01:39.542ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
42482025-12-16T03:01:39.544ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "7fa72d5d-abec-46df-bb49-ce0b5f7bb621", "content-length": "691", "date": "Tue, 16 Dec 2025 03:01:39 GMT"} })
42492025-12-16T03:01:39.544ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
42502025-12-16T03:01:39.544ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
42512025-12-16T03:01:39.544ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
42522025-12-16T03:01:39.546ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 336.201429ms iteration = 6 status = {}
42532025-12-16T03:01:39.666ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "7713641a-b613-4943-a66f-cf242b8caf46", "date": "Tue, 16 Dec 2025 03:01:39 GMT"} })
42542025-12-16T03:01:39.666ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
42552025-12-16T03:01:39.666ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
42562025-12-16T03:01:39.667ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1cea2dc-8447-410a-84df-b24c54076583", "content-length": "682", "date": "Tue, 16 Dec 2025 03:01:39 GMT"} })
42572025-12-16T03:01:39.667ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42582025-12-16T03:01:39.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42592025-12-16T03:01:39.671ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42602025-12-16T03:01:39.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42612025-12-16T03:01:39.674ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42622025-12-16T03:01:39.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42632025-12-16T03:01:39.676ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
42642025-12-16T03:01:39.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
42652025-12-16T03:01:39.684ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 473.836732ms iteration = 6 status = {}
42662025-12-16T03:01:41.060ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 280 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 6dc24069-c413-49c3-9899-5bc4bc6349ae response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
42672025-12-16T03:01:44.209ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 32 reason = Timeout
42682025-12-16T03:01:44.209ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
42692025-12-16T03:01:44.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
42702025-12-16T03:01:44.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42712025-12-16T03:01:44.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
42722025-12-16T03:01:44.214ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
42732025-12-16T03:01:44.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
42742025-12-16T03:01:44.217ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
42752025-12-16T03:01:44.217ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.099642ms iteration = 32 status = {"config_updated":false}
42762025-12-16T03:01:49.209ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 17 reason = Timeout
42772025-12-16T03:01:49.209ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 33 reason = Timeout
42782025-12-16T03:01:49.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42792025-12-16T03:01:49.209ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
42802025-12-16T03:01:49.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
42812025-12-16T03:01:49.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
42822025-12-16T03:01:49.211ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 17 reason = Timeout
42832025-12-16T03:01:49.211ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42842025-12-16T03:01:49.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42852025-12-16T03:01:49.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42862025-12-16T03:01:49.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42872025-12-16T03:01:49.213ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42882025-12-16T03:01:49.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42892025-12-16T03:01:49.214ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.645166ms iteration = 17 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
42902025-12-16T03:01:49.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
42912025-12-16T03:01:49.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42922025-12-16T03:01:49.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
42932025-12-16T03:01:49.215ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
42942025-12-16T03:01:49.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
42952025-12-16T03:01:49.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
42962025-12-16T03:01:49.217ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
42972025-12-16T03:01:49.217ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.024072ms iteration = 33 status = {"config_updated":false}
42982025-12-16T03:01:49.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42992025-12-16T03:01:49.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
43002025-12-16T03:01:49.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43012025-12-16T03:01:49.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
43022025-12-16T03:01:49.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43032025-12-16T03:01:49.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
43042025-12-16T03:01:49.233ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
43052025-12-16T03:01:49.233ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.139216ms iteration = 17 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
43062025-12-16T03:01:51.061ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 274 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 4d888be9-14d7-4f45-b264-f92958822f63 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
43072025-12-16T03:01:54.209ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 16 reason = Timeout
43082025-12-16T03:01:54.209ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 34 reason = Timeout
43092025-12-16T03:01:54.209ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
43102025-12-16T03:01:54.209ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
43112025-12-16T03:01:54.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
43122025-12-16T03:01:54.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
43132025-12-16T03:01:54.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43142025-12-16T03:01:54.211ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43152025-12-16T03:01:54.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
43162025-12-16T03:01:54.215ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
43172025-12-16T03:01:54.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
43182025-12-16T03:01:54.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
43192025-12-16T03:01:54.215ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
43202025-12-16T03:01:54.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
43212025-12-16T03:01:54.217ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
43222025-12-16T03:01:54.217ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.507506ms iteration = 34 status = {"config_updated":false}
43232025-12-16T03:01:54.217ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = fecebf81-1ff4-4296-8099-25ffd5550e29 original_time_started = 2025-12-16 03:01:10.658135 UTC
43242025-12-16T03:01:54.217ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.554486ms iteration = 16 status = {"Loaded":{"collection_id":"fecebf81-1ff4-4296-8099-25ffd5550e29","time_loaded":"2025-12-16T03:01:54.209495541Z","time_started":"2025-12-16T03:01:10.658135Z"}}
43252025-12-16T03:01:54.227ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 12 reason = Timeout
43262025-12-16T03:01:54.227ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 12 reason = Timeout
43272025-12-16T03:01:54.227ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
43282025-12-16T03:01:54.227ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
43292025-12-16T03:01:54.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
43302025-12-16T03:01:54.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
43312025-12-16T03:01:54.228ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
43322025-12-16T03:01:54.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
43332025-12-16T03:01:54.228ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43342025-12-16T03:01:54.229ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43352025-12-16T03:01:54.230ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
43362025-12-16T03:01:54.231ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
43372025-12-16T03:01:54.231ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
43382025-12-16T03:01:54.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
43392025-12-16T03:01:54.233ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.655821ms iteration = 12 status = "NoSitrep"
43402025-12-16T03:01:54.233ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
43412025-12-16T03:01:54.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
43422025-12-16T03:01:54.233ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
43432025-12-16T03:01:54.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
43442025-12-16T03:01:54.235ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43452025-12-16T03:01:54.236ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
43462025-12-16T03:01:54.237ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
43472025-12-16T03:01:54.237ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.031177ms iteration = 12 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
43482025-12-16T03:01:59.209ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 18 reason = Timeout
43492025-12-16T03:01:59.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43502025-12-16T03:01:59.209ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 35 reason = Timeout
43512025-12-16T03:01:59.209ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
43522025-12-16T03:01:59.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
43532025-12-16T03:01:59.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
43542025-12-16T03:01:59.210ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 18 reason = Timeout
43552025-12-16T03:01:59.210ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
43562025-12-16T03:01:59.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
43572025-12-16T03:01:59.212ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43582025-12-16T03:01:59.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43592025-12-16T03:01:59.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
43602025-12-16T03:01:59.215ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
43612025-12-16T03:01:59.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
43622025-12-16T03:01:59.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
43632025-12-16T03:01:59.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43642025-12-16T03:01:59.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
43652025-12-16T03:01:59.216ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
43662025-12-16T03:01:59.217ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.176561ms iteration = 35 status = {"config_updated":false}
43672025-12-16T03:01:59.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43682025-12-16T03:01:59.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
43692025-12-16T03:01:59.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43702025-12-16T03:01:59.222ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
43712025-12-16T03:01:59.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43722025-12-16T03:01:59.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
43732025-12-16T03:01:59.238ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
43742025-12-16T03:01:59.238ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.721364ms iteration = 18 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
43752025-12-16T03:01:59.249ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
43762025-12-16T03:01:59.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
43772025-12-16T03:01:59.250ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 40.416272ms iteration = 18 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
43782025-12-16T03:02:01.061ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 411 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 26e3c432-cdbb-463f-b49f-a98e1f5bfefa response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
43792025-12-16T03:02:04.210ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 36 reason = Timeout
43802025-12-16T03:02:04.210ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
43812025-12-16T03:02:04.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
43822025-12-16T03:02:04.213ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43832025-12-16T03:02:04.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
43842025-12-16T03:02:04.216ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
43852025-12-16T03:02:04.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
43862025-12-16T03:02:04.218ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
43872025-12-16T03:02:04.218ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.203574ms iteration = 36 status = {"config_updated":false}
43882025-12-16T03:02:09.168ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 4 reason = Timeout
43892025-12-16T03:02:09.168ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 4 reason = Timeout
43902025-12-16T03:02:09.168ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
43912025-12-16T03:02:09.168ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 7 reason = Timeout
43922025-12-16T03:02:09.168ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 7 reason = Timeout
43932025-12-16T03:02:09.168ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
43942025-12-16T03:02:09.168ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 4 reason = Timeout
43952025-12-16T03:02:09.168ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 7 reason = Timeout
43962025-12-16T03:02:09.168ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
43972025-12-16T03:02:09.168ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
43982025-12-16T03:02:09.168ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
43992025-12-16T03:02:09.168ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 4 reason = Timeout
44002025-12-16T03:02:09.168ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
44012025-12-16T03:02:09.168ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 6 reason = Timeout
44022025-12-16T03:02:09.168ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
44032025-12-16T03:02:09.168ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 4 reason = Timeout
44042025-12-16T03:02:09.168ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 6 reason = Timeout
44052025-12-16T03:02:09.168ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 4 reason = Timeout
44062025-12-16T03:02:09.168ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-12-16 02:52:09.168873284 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
44072025-12-16T03:02:09.168ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 5 reason = Timeout
44082025-12-16T03:02:09.168ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
44092025-12-16T03:02:09.168ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
44102025-12-16T03:02:09.168ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
44112025-12-16T03:02:09.169ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 95d92d50-37d5-447b-9982-09d309405007 step component = ExternalNetworking step id = Ensure
44122025-12-16T03:02:09.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
44132025-12-16T03:02:09.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
44142025-12-16T03:02:09.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
44152025-12-16T03:02:09.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
44162025-12-16T03:02:09.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
44172025-12-16T03:02:09.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
44182025-12-16T03:02:09.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
44192025-12-16T03:02:09.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
44202025-12-16T03:02:09.170ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1a45cd23-3d0a-43eb-adce-62c7b712baf1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1136, tv_nsec: 785671264 } })
44212025-12-16T03:02:09.170ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("49a51a59-9919-48ae-8182-4070a8430160.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6af60386-8771-4401-9d4d-e93774e519f0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 1136, tv_nsec: 785694605 } })
44222025-12-16T03:02:09.170ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
44232025-12-16T03:02:09.170ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.527089ms iteration = 4 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
44242025-12-16T03:02:09.170ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
44252025-12-16T03:02:09.170ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.496939ms iteration = 4 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
44262025-12-16T03:02:09.170ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
44272025-12-16T03:02:09.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
44282025-12-16T03:02:09.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
44292025-12-16T03:02:09.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
44302025-12-16T03:02:09.175ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1136, tv_nsec: 791363636 } })
44312025-12-16T03:02:09.177ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
44322025-12-16T03:02:09.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
44332025-12-16T03:02:09.178ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44342025-12-16T03:02:09.180ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-12-16 02:52:09.168873284 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
44352025-12-16T03:02:09.181ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 12.109734ms iteration = 4 status = {"expiration":"2025-12-16T02:52:09.168873284Z","pruned":[]}
44362025-12-16T03:02:09.181ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1136, tv_nsec: 797232841 } })
44372025-12-16T03:02:09.181ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
44382025-12-16T03:02:09.182ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
44392025-12-16T03:02:09.182ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1136, tv_nsec: 798096072 } })
44402025-12-16T03:02:09.182ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
44412025-12-16T03:02:09.182ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
44422025-12-16T03:02:09.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
44432025-12-16T03:02:09.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
44442025-12-16T03:02:09.186ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
44452025-12-16T03:02:09.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
44462025-12-16T03:02:09.186ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44472025-12-16T03:02:09.189ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44482025-12-16T03:02:09.190ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 21.944968ms iteration = 7 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
44492025-12-16T03:02:09.192ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
44502025-12-16T03:02:09.197ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
44512025-12-16T03:02:09.197ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
44522025-12-16T03:02:09.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
44532025-12-16T03:02:09.198ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
44542025-12-16T03:02:09.209ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 11 reason = Timeout
44552025-12-16T03:02:09.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44562025-12-16T03:02:09.209ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 19 reason = Timeout
44572025-12-16T03:02:09.209ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 37 reason = Timeout
44582025-12-16T03:02:09.209ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
44592025-12-16T03:02:09.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
44602025-12-16T03:02:09.209ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 17 reason = Timeout
44612025-12-16T03:02:09.209ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
44622025-12-16T03:02:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
44632025-12-16T03:02:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44642025-12-16T03:02:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
44652025-12-16T03:02:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
44662025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 3 reason = Timeout
44672025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 12 reason = Timeout
44682025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
44692025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 4 reason = Timeout
44702025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
44712025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 10 reason = Timeout
44722025-12-16T03:02:09.211ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
44732025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
44742025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44752025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 7 reason = Timeout
44762025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
44772025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 7 reason = Timeout
44782025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
44792025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 7 reason = Timeout
44802025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
44812025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 7 reason = Timeout
44822025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44832025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 7 reason = Timeout
44842025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 19 reason = Timeout
44852025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
44862025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
44872025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 10 reason = Timeout
44882025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
44892025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 4 reason = Timeout
44902025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
44912025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 4 reason = Timeout
44922025-12-16T03:02:09.211ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
44932025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
44942025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 7 reason = Timeout
44952025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
44962025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
44972025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44982025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44992025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
45002025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
45012025-12-16T03:02:09.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45022025-12-16T03:02:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
45032025-12-16T03:02:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
45042025-12-16T03:02:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45052025-12-16T03:02:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
45062025-12-16T03:02:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
45072025-12-16T03:02:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
45082025-12-16T03:02:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
45092025-12-16T03:02:09.212ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
45102025-12-16T03:02:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
45112025-12-16T03:02:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
45122025-12-16T03:02:09.212ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
45132025-12-16T03:02:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
45142025-12-16T03:02:09.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
45152025-12-16T03:02:09.213ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
45162025-12-16T03:02:09.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
45172025-12-16T03:02:09.213ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45182025-12-16T03:02:09.213ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
45192025-12-16T03:02:09.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
45202025-12-16T03:02:09.213ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
45212025-12-16T03:02:09.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
45222025-12-16T03:02:09.214ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45232025-12-16T03:02:09.214ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45242025-12-16T03:02:09.214ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
45252025-12-16T03:02:09.215ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45262025-12-16T03:02:09.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
45272025-12-16T03:02:09.215ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45282025-12-16T03:02:09.216ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45292025-12-16T03:02:09.217ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
45302025-12-16T03:02:09.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45312025-12-16T03:02:09.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45322025-12-16T03:02:09.218ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45332025-12-16T03:02:09.218ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
45342025-12-16T03:02:09.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
45352025-12-16T03:02:09.219ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
45362025-12-16T03:02:09.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
45372025-12-16T03:02:09.220ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
45382025-12-16T03:02:09.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
45392025-12-16T03:02:09.220ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
45402025-12-16T03:02:09.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
45412025-12-16T03:02:09.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45422025-12-16T03:02:09.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
45432025-12-16T03:02:09.220ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
45442025-12-16T03:02:09.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
45452025-12-16T03:02:09.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
45462025-12-16T03:02:09.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
45472025-12-16T03:02:09.221ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
45482025-12-16T03:02:09.221ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
45492025-12-16T03:02:09.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
45502025-12-16T03:02:09.221ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
45512025-12-16T03:02:09.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
45522025-12-16T03:02:09.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
45532025-12-16T03:02:09.222ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
45542025-12-16T03:02:09.222ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
45552025-12-16T03:02:09.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
45562025-12-16T03:02:09.222ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45572025-12-16T03:02:09.223ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
45582025-12-16T03:02:09.223ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45592025-12-16T03:02:09.223ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45602025-12-16T03:02:09.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45612025-12-16T03:02:09.225ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
45622025-12-16T03:02:09.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
45632025-12-16T03:02:09.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
45642025-12-16T03:02:09.226ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
45652025-12-16T03:02:09.226ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 16.870363ms iteration = 37 status = {"config_updated":false}
45662025-12-16T03:02:09.226ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
45672025-12-16T03:02:09.227ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
45682025-12-16T03:02:09.227ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
45692025-12-16T03:02:09.227ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 13 reason = Timeout
45702025-12-16T03:02:09.227ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
45712025-12-16T03:02:09.227ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 7 reason = Timeout
45722025-12-16T03:02:09.227ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
45732025-12-16T03:02:09.227ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 23.56\u{b5}s iteration = 13 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
45742025-12-16T03:02:09.227ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 4 reason = Timeout
45752025-12-16T03:02:09.227ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
45762025-12-16T03:02:09.227ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 7 reason = Timeout
45772025-12-16T03:02:09.227ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 4 reason = Timeout
45782025-12-16T03:02:09.227ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
45792025-12-16T03:02:09.227ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
45802025-12-16T03:02:09.227ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
45812025-12-16T03:02:09.227ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 7 reason = Timeout
45822025-12-16T03:02:09.227ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
45832025-12-16T03:02:09.227ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 7 reason = Timeout
45842025-12-16T03:02:09.227ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 12.171\u{b5}s iteration = 7 status = {"disabled":true,"errors":[],"sps":[]}
45852025-12-16T03:02:09.227ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 13 reason = Timeout
45862025-12-16T03:02:09.227ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
45872025-12-16T03:02:09.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
45882025-12-16T03:02:09.227ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45892025-12-16T03:02:09.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
45902025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
45912025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 7 reason = Timeout
45922025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
45932025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
45942025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 4 reason = Timeout
45952025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
45962025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
45972025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 4 reason = Timeout
45982025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
45992025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 13 reason = Timeout
46002025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
46012025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
46022025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
46032025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 7 reason = Timeout
46042025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
46052025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
46062025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
46072025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
46082025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
46092025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
46102025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
46112025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
46122025-12-16T03:02:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
46132025-12-16T03:02:09.231ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
46142025-12-16T03:02:09.231ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
46152025-12-16T03:02:09.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
46162025-12-16T03:02:09.231ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46172025-12-16T03:02:09.233ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46182025-12-16T03:02:09.235ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
46192025-12-16T03:02:09.235ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
46202025-12-16T03:02:09.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
46212025-12-16T03:02:09.237ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
46222025-12-16T03:02:09.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
46232025-12-16T03:02:09.239ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
46242025-12-16T03:02:09.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46252025-12-16T03:02:09.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
46262025-12-16T03:02:09.243ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
46272025-12-16T03:02:09.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
46282025-12-16T03:02:09.243ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1136, tv_nsec: 859471439 } })
46292025-12-16T03:02:09.244ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
46302025-12-16T03:02:09.244ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1136, tv_nsec: 860588494 } })
46312025-12-16T03:02:09.244ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
46322025-12-16T03:02:09.249ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = fecebf81-1ff4-4296-8099-25ffd5550e29 original_time_started = 2025-12-16 03:01:10.658135 UTC
46332025-12-16T03:02:09.249ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 40.017697ms iteration = 17 status = {"Loaded":{"collection_id":"fecebf81-1ff4-4296-8099-25ffd5550e29","time_loaded":"2025-12-16T03:02:09.209671460Z","time_started":"2025-12-16T03:01:10.658135Z"}}
46342025-12-16T03:02:09.250ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46352025-12-16T03:02:09.253ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
46362025-12-16T03:02:09.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
46372025-12-16T03:02:09.253ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
46382025-12-16T03:02:09.254ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
46392025-12-16T03:02:09.254ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
46402025-12-16T03:02:09.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
46412025-12-16T03:02:09.254ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
46422025-12-16T03:02:09.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
46432025-12-16T03:02:09.259ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
46442025-12-16T03:02:09.266ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
46452025-12-16T03:02:09.269ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
46462025-12-16T03:02:09.273ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
46472025-12-16T03:02:09.290ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46482025-12-16T03:02:09.290ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
46492025-12-16T03:02:09.291ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 95d92d50-37d5-447b-9982-09d309405007 step component = SledList step id = Fetch
46502025-12-16T03:02:09.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
46512025-12-16T03:02:09.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46522025-12-16T03:02:09.295ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46532025-12-16T03:02:09.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
46542025-12-16T03:02:09.296ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 68.837912ms iteration = 4 status = {"errors":[],"found_port_ok":[]}
46552025-12-16T03:02:09.300ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
46562025-12-16T03:02:09.300ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 72.55032ms iteration = 4 status = {"by_rx":{},"error":null}
46572025-12-16T03:02:09.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
46582025-12-16T03:02:09.307ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46592025-12-16T03:02:09.311ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
46602025-12-16T03:02:09.311ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
46612025-12-16T03:02:09.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
46622025-12-16T03:02:09.314ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46632025-12-16T03:02:09.316ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
46642025-12-16T03:02:09.316ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
46652025-12-16T03:02:09.318ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "92fad7c0-cf39-4831-bf25-87da9345a284", "content-length": "26", "date": "Tue, 16 Dec 2025 03:02:09 GMT"} })
46662025-12-16T03:02:09.318ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
46672025-12-16T03:02:09.318ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
46682025-12-16T03:02:09.318ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
46692025-12-16T03:02:09.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
46702025-12-16T03:02:09.319ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
46712025-12-16T03:02:09.321ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1136, tv_nsec: 937163444 } })
46722025-12-16T03:02:09.324ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46732025-12-16T03:02:09.326ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
46742025-12-16T03:02:09.326ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 115.272791ms iteration = 7 status = 0
46752025-12-16T03:02:09.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
46762025-12-16T03:02:09.338ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
46772025-12-16T03:02:09.338ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
46782025-12-16T03:02:09.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
46792025-12-16T03:02:09.339ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46802025-12-16T03:02:09.342ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46812025-12-16T03:02:09.343ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46822025-12-16T03:02:09.343ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
46832025-12-16T03:02:09.343ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
46842025-12-16T03:02:09.343ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
46852025-12-16T03:02:09.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
46862025-12-16T03:02:09.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
46872025-12-16T03:02:09.344ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
46882025-12-16T03:02:09.344ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
46892025-12-16T03:02:09.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
46902025-12-16T03:02:09.345ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
46912025-12-16T03:02:09.345ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46922025-12-16T03:02:09.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
46932025-12-16T03:02:09.347ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
46942025-12-16T03:02:09.347ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
46952025-12-16T03:02:09.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
46962025-12-16T03:02:09.348ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46972025-12-16T03:02:09.348ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
46982025-12-16T03:02:09.349ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
46992025-12-16T03:02:09.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
47002025-12-16T03:02:09.351ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47012025-12-16T03:02:09.351ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47022025-12-16T03:02:09.351ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 123.254211ms iteration = 13 status = "NoSitrep"
47032025-12-16T03:02:09.351ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
47042025-12-16T03:02:09.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
47052025-12-16T03:02:09.351ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
47062025-12-16T03:02:09.351ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
47072025-12-16T03:02:09.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
47082025-12-16T03:02:09.352ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
47092025-12-16T03:02:09.353ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
47102025-12-16T03:02:09.353ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
47112025-12-16T03:02:09.353ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 141.688476ms iteration = 4 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
47122025-12-16T03:02:09.354ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 143.271856ms iteration = 3 status = {"nsuccess":5}
47132025-12-16T03:02:09.356ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
47142025-12-16T03:02:09.356ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47152025-12-16T03:02:09.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47162025-12-16T03:02:09.357ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
47172025-12-16T03:02:09.357ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 146.05131ms iteration = 19 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
47182025-12-16T03:02:09.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47192025-12-16T03:02:09.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47202025-12-16T03:02:09.358ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
47212025-12-16T03:02:09.358ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 146.81593ms iteration = 7 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
47222025-12-16T03:02:09.359ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
47232025-12-16T03:02:09.360ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47242025-12-16T03:02:09.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
47252025-12-16T03:02:09.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
47262025-12-16T03:02:09.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47272025-12-16T03:02:09.365ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
47282025-12-16T03:02:09.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
47292025-12-16T03:02:09.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
47302025-12-16T03:02:09.365ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 137.869257ms iteration = 4 status = {"dispatched":[],"errors":[],"glob_version":"212.0.0","globs_reprocessed":{},"no_receivers":[]}
47312025-12-16T03:02:09.365ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
47322025-12-16T03:02:09.365ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
47332025-12-16T03:02:09.365ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 153.916581ms iteration = 4 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
47342025-12-16T03:02:09.367ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47352025-12-16T03:02:09.368ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
47362025-12-16T03:02:09.368ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47372025-12-16T03:02:09.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
47382025-12-16T03:02:09.370ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 142.8269ms iteration = 7 status = {"errors":[],"finish_invoked_ok":[]}
47392025-12-16T03:02:09.371ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 159.643822ms iteration = 7 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
47402025-12-16T03:02:09.371ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
47412025-12-16T03:02:09.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47422025-12-16T03:02:09.372ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
47432025-12-16T03:02:09.372ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 144.708484ms iteration = 13 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
47442025-12-16T03:02:09.374ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
47452025-12-16T03:02:09.376ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
47462025-12-16T03:02:09.376ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47472025-12-16T03:02:09.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47482025-12-16T03:02:09.377ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
47492025-12-16T03:02:09.377ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
47502025-12-16T03:02:09.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
47512025-12-16T03:02:09.387ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
47522025-12-16T03:02:09.390ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
47532025-12-16T03:02:09.390ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
47542025-12-16T03:02:09.390ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
47552025-12-16T03:02:09.390ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
47562025-12-16T03:02:09.390ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 221.70993ms iteration = 7 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
47572025-12-16T03:02:09.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
47582025-12-16T03:02:09.391ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 163.524232ms iteration = 7 status = {"errors":[],"garbage_collect_requested":[]}
47592025-12-16T03:02:09.402ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47602025-12-16T03:02:09.406ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 178.265079ms iteration = 7 status = {"errors":[],"requests_created_ok":[]}
47612025-12-16T03:02:09.408ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
47622025-12-16T03:02:09.408ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
47632025-12-16T03:02:09.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
47642025-12-16T03:02:09.412ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
47652025-12-16T03:02:09.412ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
47662025-12-16T03:02:09.412ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
47672025-12-16T03:02:09.412ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
47682025-12-16T03:02:09.412ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
47692025-12-16T03:02:09.412ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
47702025-12-16T03:02:09.412ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
47712025-12-16T03:02:09.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
47722025-12-16T03:02:09.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
47732025-12-16T03:02:09.418ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47742025-12-16T03:02:09.419ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
47752025-12-16T03:02:09.419ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
47762025-12-16T03:02:09.419ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c3b97da-03a3-4de2-8de1-6f6a01979c3f", "content-length": "26", "date": "Tue, 16 Dec 2025 03:02:09 GMT"} })
47772025-12-16T03:02:09.419ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47782025-12-16T03:02:09.419ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
47792025-12-16T03:02:09.419ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
47802025-12-16T03:02:09.419ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 209.892009ms iteration = 19 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
47812025-12-16T03:02:09.422ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
47822025-12-16T03:02:09.425ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
47832025-12-16T03:02:09.425ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
47842025-12-16T03:02:09.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
47852025-12-16T03:02:09.429ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
47862025-12-16T03:02:09.429ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
47872025-12-16T03:02:09.429ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-12-16 02:59:18.154440 UTC dns_group = external generation = 2
47882025-12-16T03:02:09.429ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 260.421769ms iteration = 4 status = {"generation":2}
47892025-12-16T03:02:09.431ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
47902025-12-16T03:02:09.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
47912025-12-16T03:02:09.432ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
47922025-12-16T03:02:09.432ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
47932025-12-16T03:02:09.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
47942025-12-16T03:02:09.435ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
47952025-12-16T03:02:09.437ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 226.178405ms iteration = 10 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
47962025-12-16T03:02:09.437ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
47972025-12-16T03:02:09.437ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
47982025-12-16T03:02:09.437ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
47992025-12-16T03:02:09.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
48002025-12-16T03:02:09.440ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48012025-12-16T03:02:09.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48022025-12-16T03:02:09.441ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48032025-12-16T03:02:09.441ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48042025-12-16T03:02:09.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48052025-12-16T03:02:09.445ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
48062025-12-16T03:02:09.445ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
48072025-12-16T03:02:09.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
48082025-12-16T03:02:09.447ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
48092025-12-16T03:02:09.447ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
48102025-12-16T03:02:09.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
48112025-12-16T03:02:09.448ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
48122025-12-16T03:02:09.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48132025-12-16T03:02:09.453ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 225.213133ms iteration = 7 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
48142025-12-16T03:02:09.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48152025-12-16T03:02:09.454ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
48162025-12-16T03:02:09.454ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
48172025-12-16T03:02:09.454ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-12-16 02:59:16.791641 UTC dns_group = internal generation = 1
48182025-12-16T03:02:09.454ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 285.71864ms iteration = 4 status = {"generation":1}
48192025-12-16T03:02:09.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48202025-12-16T03:02:09.459ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 290.50536ms iteration = 7 status = {}
48212025-12-16T03:02:09.459ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
48222025-12-16T03:02:09.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48232025-12-16T03:02:09.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48242025-12-16T03:02:09.463ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
48252025-12-16T03:02:09.463ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
48262025-12-16T03:02:09.463ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
48272025-12-16T03:02:09.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
48282025-12-16T03:02:09.463ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-12-16T02:59:05.020237Z, time_modified: 2025-12-16T02:59:05.020237Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }) }
48292025-12-16T03:02:09.463ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 295.24444ms iteration = 4 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]}},"default_endpoint":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
48302025-12-16T03:02:09.470ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
48312025-12-16T03:02:09.470ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
48322025-12-16T03:02:09.470ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 258.514475ms iteration = 7 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
48332025-12-16T03:02:09.508ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 280.758087ms iteration = 7 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
48342025-12-16T03:02:09.513ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
48352025-12-16T03:02:09.513ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
48362025-12-16T03:02:09.514ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a888bcb1-8710-41dc-9138-453efbda0c02", "content-length": "372", "date": "Tue, 16 Dec 2025 03:02:09 GMT"} })
48372025-12-16T03:02:09.514ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48382025-12-16T03:02:09.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48392025-12-16T03:02:09.516ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
48402025-12-16T03:02:09.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
48412025-12-16T03:02:09.524ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
48422025-12-16T03:02:09.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
48432025-12-16T03:02:09.528ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48442025-12-16T03:02:09.536ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
48452025-12-16T03:02:09.536ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
48462025-12-16T03:02:09.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
48472025-12-16T03:02:09.541ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 95d92d50-37d5-447b-9982-09d309405007 step component = DeployNexusRecords step id = Ensure
48482025-12-16T03:02:09.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48492025-12-16T03:02:09.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48502025-12-16T03:02:09.544ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
48512025-12-16T03:02:09.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48522025-12-16T03:02:09.544ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "e2ff523b-2cb0-4a3b-94c6-81f8ed4d02dc", "date": "Tue, 16 Dec 2025 03:02:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
48532025-12-16T03:02:09.544ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
48542025-12-16T03:02:09.545ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
48552025-12-16T03:02:09.545ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
48562025-12-16T03:02:09.545ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "d159ab2e-f238-4003-b57b-046519375301", "date": "Tue, 16 Dec 2025 03:02:09 GMT"} })
48572025-12-16T03:02:09.545ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48582025-12-16T03:02:09.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48592025-12-16T03:02:09.546ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "d19da073-7ef7-43b2-95a6-1e1c83bbd847", "date": "Tue, 16 Dec 2025 03:02:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
48602025-12-16T03:02:09.546ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
48612025-12-16T03:02:09.547ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
48622025-12-16T03:02:09.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48632025-12-16T03:02:09.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48642025-12-16T03:02:09.548ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
48652025-12-16T03:02:09.548ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
48662025-12-16T03:02:09.549ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48672025-12-16T03:02:09.549ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 380.114226ms iteration = 6 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
48682025-12-16T03:02:09.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48692025-12-16T03:02:09.549ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48702025-12-16T03:02:09.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48712025-12-16T03:02:09.551ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48722025-12-16T03:02:09.553ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48732025-12-16T03:02:09.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48742025-12-16T03:02:09.555ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48752025-12-16T03:02:09.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48762025-12-16T03:02:09.556ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
48772025-12-16T03:02:09.558ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48782025-12-16T03:02:09.558ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1137, tv_nsec: 174535731 } })
48792025-12-16T03:02:09.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48802025-12-16T03:02:09.559ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
48812025-12-16T03:02:09.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
48822025-12-16T03:02:09.562ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48832025-12-16T03:02:09.565ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
48842025-12-16T03:02:09.565ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
48852025-12-16T03:02:09.566ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "b11fbd32-9384-4d44-98d4-29333671447e", "date": "Tue, 16 Dec 2025 03:02:09 GMT"} })
48862025-12-16T03:02:09.566ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
48872025-12-16T03:02:09.566ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
48882025-12-16T03:02:09.566ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48892025-12-16T03:02:09.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
48902025-12-16T03:02:09.568ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 357.247016ms iteration = 10 status = {}
48912025-12-16T03:02:09.570ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48922025-12-16T03:02:09.572ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
48932025-12-16T03:02:09.572ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48942025-12-16T03:02:09.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48952025-12-16T03:02:09.573ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
48962025-12-16T03:02:09.573ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
48972025-12-16T03:02:09.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
48982025-12-16T03:02:09.575ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48992025-12-16T03:02:09.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49002025-12-16T03:02:09.578ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49012025-12-16T03:02:09.583ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
49022025-12-16T03:02:09.583ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49032025-12-16T03:02:09.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49042025-12-16T03:02:09.586ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49052025-12-16T03:02:09.591ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 379.928893ms iteration = 12 status = {"physical_disks_added":0}
49062025-12-16T03:02:09.591ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
49072025-12-16T03:02:09.591ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49082025-12-16T03:02:09.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49092025-12-16T03:02:09.594ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49102025-12-16T03:02:09.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49112025-12-16T03:02:09.595ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49122025-12-16T03:02:09.600ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
49132025-12-16T03:02:09.600ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49142025-12-16T03:02:09.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49152025-12-16T03:02:09.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49162025-12-16T03:02:09.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49172025-12-16T03:02:09.611ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49182025-12-16T03:02:09.616ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
49192025-12-16T03:02:09.616ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49202025-12-16T03:02:09.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49212025-12-16T03:02:09.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49222025-12-16T03:02:09.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49232025-12-16T03:02:09.622ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49242025-12-16T03:02:09.624ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 95d92d50-37d5-447b-9982-09d309405007 step component = SledAgent step id = Ensure
49252025-12-16T03:02:09.625ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
49262025-12-16T03:02:09.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49272025-12-16T03:02:09.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49282025-12-16T03:02:09.657ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49292025-12-16T03:02:09.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49302025-12-16T03:02:09.661ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49312025-12-16T03:02:09.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49322025-12-16T03:02:09.664ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49332025-12-16T03:02:09.664ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
49342025-12-16T03:02:09.665ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "70e17157-5237-4fe8-9a48-cde97e93bd96", "date": "Tue, 16 Dec 2025 03:02:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
49352025-12-16T03:02:09.665ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
49362025-12-16T03:02:09.667ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1137, tv_nsec: 283979627 } })
49372025-12-16T03:02:09.668ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
49382025-12-16T03:02:09.668ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "1713d6f8-29d3-4bc9-931c-6b3e13a26631", "date": "Tue, 16 Dec 2025 03:02:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
49392025-12-16T03:02:09.668ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
49402025-12-16T03:02:09.670ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
49412025-12-16T03:02:09.670ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49422025-12-16T03:02:09.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49432025-12-16T03:02:09.671ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "5f28a96c-b19b-47c3-a249-e7b7d7e67273", "date": "Tue, 16 Dec 2025 03:02:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
49442025-12-16T03:02:09.671ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
49452025-12-16T03:02:09.674ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
49462025-12-16T03:02:09.674ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 505.313681ms iteration = 6 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
49472025-12-16T03:02:09.677ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "532d6472-613f-4366-b5fd-c990220d22d9", "date": "Tue, 16 Dec 2025 03:02:09 GMT"} })
49482025-12-16T03:02:09.677ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
49492025-12-16T03:02:09.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49502025-12-16T03:02:09.679ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
49512025-12-16T03:02:09.679ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 452.328511ms iteration = 4 status = {"errors":[],"probes_by_sled":{"b1c76b61-acee-4be6-b582-973c938d6b6e":0}}
49522025-12-16T03:02:09.679ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 11 reason = Dependency
49532025-12-16T03:02:09.679ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
49542025-12-16T03:02:09.679ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49552025-12-16T03:02:09.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49562025-12-16T03:02:09.683ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49572025-12-16T03:02:09.683ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
49582025-12-16T03:02:09.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49592025-12-16T03:02:09.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49602025-12-16T03:02:09.686ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
49612025-12-16T03:02:09.686ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49622025-12-16T03:02:09.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49632025-12-16T03:02:09.688ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49642025-12-16T03:02:09.689ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49652025-12-16T03:02:09.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49662025-12-16T03:02:09.692ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49672025-12-16T03:02:09.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49682025-12-16T03:02:09.693ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
49692025-12-16T03:02:09.693ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49702025-12-16T03:02:09.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49712025-12-16T03:02:09.694ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49722025-12-16T03:02:09.699ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
49732025-12-16T03:02:09.699ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49742025-12-16T03:02:09.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49752025-12-16T03:02:09.701ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49762025-12-16T03:02:09.701ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
49772025-12-16T03:02:09.704ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
49782025-12-16T03:02:09.704ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49792025-12-16T03:02:09.705ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49802025-12-16T03:02:09.708ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49812025-12-16T03:02:09.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49822025-12-16T03:02:09.709ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49832025-12-16T03:02:09.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49842025-12-16T03:02:09.711ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49852025-12-16T03:02:09.712ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "06141b0c-d91f-4996-aec4-745eb90bb089", "date": "Tue, 16 Dec 2025 03:02:09 GMT"} }) sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
49862025-12-16T03:02:09.714ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 95d92d50-37d5-447b-9982-09d309405007 step component = FirewallRules step id = Ensure
49872025-12-16T03:02:09.714ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49882025-12-16T03:02:09.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49892025-12-16T03:02:09.715ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
49902025-12-16T03:02:09.716ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49912025-12-16T03:02:09.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49922025-12-16T03:02:09.719ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49932025-12-16T03:02:09.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49942025-12-16T03:02:09.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49952025-12-16T03:02:09.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49962025-12-16T03:02:09.724ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49972025-12-16T03:02:09.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49982025-12-16T03:02:09.725ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
49992025-12-16T03:02:09.725ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 514.318986ms iteration = 4 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
50002025-12-16T03:02:09.727ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50012025-12-16T03:02:09.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50022025-12-16T03:02:09.729ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50032025-12-16T03:02:09.730ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50042025-12-16T03:02:09.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50052025-12-16T03:02:09.732ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
50062025-12-16T03:02:09.732ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50072025-12-16T03:02:09.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50082025-12-16T03:02:09.737ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
50092025-12-16T03:02:09.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50102025-12-16T03:02:09.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50112025-12-16T03:02:09.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50122025-12-16T03:02:09.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50132025-12-16T03:02:09.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50142025-12-16T03:02:09.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50152025-12-16T03:02:09.742ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50162025-12-16T03:02:09.746ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
50172025-12-16T03:02:09.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50182025-12-16T03:02:09.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50192025-12-16T03:02:09.747ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
50202025-12-16T03:02:09.747ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
50212025-12-16T03:02:09.748ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
50222025-12-16T03:02:09.748ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
50232025-12-16T03:02:09.748ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
50242025-12-16T03:02:09.748ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
50252025-12-16T03:02:09.748ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
50262025-12-16T03:02:09.748ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
50272025-12-16T03:02:09.748ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
50282025-12-16T03:02:09.748ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
50292025-12-16T03:02:09.749ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
50302025-12-16T03:02:09.749ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50312025-12-16T03:02:09.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50322025-12-16T03:02:09.749ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50332025-12-16T03:02:09.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50342025-12-16T03:02:09.751ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
50352025-12-16T03:02:09.751ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
50362025-12-16T03:02:09.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50372025-12-16T03:02:09.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50382025-12-16T03:02:09.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50392025-12-16T03:02:09.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50402025-12-16T03:02:09.757ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
50412025-12-16T03:02:09.757ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0884024-7274-44c3-aaf1-f95fc0630f6f", "content-length": "691", "date": "Tue, 16 Dec 2025 03:02:09 GMT"} })
50422025-12-16T03:02:09.757ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
50432025-12-16T03:02:09.757ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
50442025-12-16T03:02:09.757ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
50452025-12-16T03:02:09.759ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 548.214685ms iteration = 7 status = {}
50462025-12-16T03:02:09.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50472025-12-16T03:02:09.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50482025-12-16T03:02:09.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50492025-12-16T03:02:09.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50502025-12-16T03:02:09.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50512025-12-16T03:02:09.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50522025-12-16T03:02:09.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50532025-12-16T03:02:09.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
50542025-12-16T03:02:09.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50552025-12-16T03:02:09.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50562025-12-16T03:02:09.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50572025-12-16T03:02:09.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50582025-12-16T03:02:09.771ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50592025-12-16T03:02:09.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50602025-12-16T03:02:09.771ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50612025-12-16T03:02:09.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50622025-12-16T03:02:09.773ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50632025-12-16T03:02:09.773ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50642025-12-16T03:02:09.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50652025-12-16T03:02:09.777ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50662025-12-16T03:02:09.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50672025-12-16T03:02:09.778ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
50682025-12-16T03:02:09.781ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50692025-12-16T03:02:09.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50702025-12-16T03:02:09.837ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50712025-12-16T03:02:09.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50722025-12-16T03:02:09.839ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50732025-12-16T03:02:09.839ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50742025-12-16T03:02:09.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50752025-12-16T03:02:09.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50762025-12-16T03:02:09.841ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50772025-12-16T03:02:09.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50782025-12-16T03:02:09.842ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50792025-12-16T03:02:09.843ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50802025-12-16T03:02:09.843ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
50812025-12-16T03:02:09.843ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
50822025-12-16T03:02:09.844ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7d0eb6c-5c08-4c85-80fa-51d8c1544f83", "content-length": "372", "date": "Tue, 16 Dec 2025 03:02:09 GMT"} })
50832025-12-16T03:02:09.844ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
50842025-12-16T03:02:09.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
50852025-12-16T03:02:09.846ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
50862025-12-16T03:02:09.846ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50872025-12-16T03:02:09.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50882025-12-16T03:02:09.847ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50892025-12-16T03:02:09.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50902025-12-16T03:02:09.849ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
50912025-12-16T03:02:09.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50922025-12-16T03:02:09.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50932025-12-16T03:02:09.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50942025-12-16T03:02:09.850ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50952025-12-16T03:02:09.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50962025-12-16T03:02:09.852ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
50972025-12-16T03:02:09.852ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
50982025-12-16T03:02:09.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
50992025-12-16T03:02:09.855ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
51002025-12-16T03:02:09.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
51012025-12-16T03:02:09.856ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51022025-12-16T03:02:09.860ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
51032025-12-16T03:02:09.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
51042025-12-16T03:02:09.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
51052025-12-16T03:02:09.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
51062025-12-16T03:02:09.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
51072025-12-16T03:02:09.863ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51082025-12-16T03:02:09.866ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
51092025-12-16T03:02:09.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
51102025-12-16T03:02:09.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
51112025-12-16T03:02:09.899ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51122025-12-16T03:02:09.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51132025-12-16T03:02:09.901ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51142025-12-16T03:02:09.906ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
51152025-12-16T03:02:09.906ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
51162025-12-16T03:02:09.908ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "56242222-ea33-48c8-9e6b-cfa53ca01a25", "date": "Tue, 16 Dec 2025 03:02:09 GMT"} })
51172025-12-16T03:02:09.908ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
51182025-12-16T03:02:09.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
51192025-12-16T03:02:09.909ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
51202025-12-16T03:02:09.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51212025-12-16T03:02:09.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51222025-12-16T03:02:09.911ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
51232025-12-16T03:02:09.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
51242025-12-16T03:02:09.912ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51252025-12-16T03:02:09.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51262025-12-16T03:02:09.914ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
51272025-12-16T03:02:09.914ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51282025-12-16T03:02:09.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
51292025-12-16T03:02:09.914ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
51302025-12-16T03:02:09.914ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
51312025-12-16T03:02:09.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51322025-12-16T03:02:09.914ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 26a46e7c-f23c-4ffa-8497-c061a8648c1d parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
51332025-12-16T03:02:09.914ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
51342025-12-16T03:02:09.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
51352025-12-16T03:02:09.915ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51362025-12-16T03:02:09.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51372025-12-16T03:02:09.916ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51382025-12-16T03:02:09.917ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
51392025-12-16T03:02:09.917ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
51402025-12-16T03:02:09.917ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "48e35453-d713-463b-95ac-abd15b798bce", "date": "Tue, 16 Dec 2025 03:02:09 GMT"} })
51412025-12-16T03:02:09.918ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51422025-12-16T03:02:09.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51432025-12-16T03:02:09.919ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 239.280132ms iteration = 11 status = {}
51442025-12-16T03:02:09.919ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
51452025-12-16T03:02:09.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
51462025-12-16T03:02:09.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
51472025-12-16T03:02:09.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51482025-12-16T03:02:09.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51492025-12-16T03:02:09.922ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51502025-12-16T03:02:09.922ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
51512025-12-16T03:02:09.922ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
51522025-12-16T03:02:09.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51532025-12-16T03:02:09.922ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 712.870432ms iteration = 11 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
51542025-12-16T03:02:09.923ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51552025-12-16T03:02:09.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51562025-12-16T03:02:09.925ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51572025-12-16T03:02:09.930ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
51582025-12-16T03:02:09.930ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51592025-12-16T03:02:09.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51602025-12-16T03:02:09.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51612025-12-16T03:02:09.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51622025-12-16T03:02:09.933ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51632025-12-16T03:02:09.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51642025-12-16T03:02:09.935ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51652025-12-16T03:02:09.940ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
51662025-12-16T03:02:09.940ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51672025-12-16T03:02:09.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51682025-12-16T03:02:09.949ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "01b04a8c-0d0c-40cd-97ea-9f589d0eba63", "date": "Tue, 16 Dec 2025 03:02:09 GMT"} })
51692025-12-16T03:02:09.949ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
51702025-12-16T03:02:09.949ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
51712025-12-16T03:02:09.951ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], Name("nexus"): [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
51722025-12-16T03:02:09.951ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51732025-12-16T03:02:09.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51742025-12-16T03:02:09.953ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51752025-12-16T03:02:09.955ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
51762025-12-16T03:02:09.955ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51772025-12-16T03:02:09.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51782025-12-16T03:02:09.956ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "14b51899-9421-4922-a4c7-cfe5172afd98", "content-length": "682", "date": "Tue, 16 Dec 2025 03:02:09 GMT"} })
51792025-12-16T03:02:09.956ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51802025-12-16T03:02:09.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51812025-12-16T03:02:09.957ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
51822025-12-16T03:02:09.957ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
51832025-12-16T03:02:09.957ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
51842025-12-16T03:02:09.960ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51852025-12-16T03:02:09.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51862025-12-16T03:02:09.962ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51872025-12-16T03:02:09.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51882025-12-16T03:02:09.964ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51892025-12-16T03:02:09.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51902025-12-16T03:02:09.971ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 760.285073ms iteration = 7 status = {}
51912025-12-16T03:02:10.101ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), time_created: 2025-12-16T02:59:15.195809Z, time_modified: 2025-12-16T02:59:15.195809Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
51922025-12-16T03:02:10.101ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
51932025-12-16T03:02:10.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51942025-12-16T03:02:10.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51952025-12-16T03:02:10.133ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51962025-12-16T03:02:10.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51972025-12-16T03:02:10.135ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51982025-12-16T03:02:10.138ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
51992025-12-16T03:02:10.138ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
52002025-12-16T03:02:10.139ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "6a97b544-75db-4a46-a481-2b589989c36a", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
52012025-12-16T03:02:10.140ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
52022025-12-16T03:02:10.140ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 95d92d50-37d5-447b-9982-09d309405007 step component = Dns step id = Ensure
52032025-12-16T03:02:10.140ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
52042025-12-16T03:02:10.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52052025-12-16T03:02:10.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52062025-12-16T03:02:10.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52072025-12-16T03:02:10.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52082025-12-16T03:02:10.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52092025-12-16T03:02:10.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
52102025-12-16T03:02:10.150ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
52112025-12-16T03:02:10.150ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
52122025-12-16T03:02:10.150ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
52132025-12-16T03:02:10.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52142025-12-16T03:02:10.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52152025-12-16T03:02:10.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52162025-12-16T03:02:10.155ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
52172025-12-16T03:02:10.155ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52182025-12-16T03:02:10.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52192025-12-16T03:02:10.208ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
52202025-12-16T03:02:10.208ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
52212025-12-16T03:02:10.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52222025-12-16T03:02:10.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52232025-12-16T03:02:10.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52242025-12-16T03:02:10.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
52252025-12-16T03:02:10.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52262025-12-16T03:02:10.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52272025-12-16T03:02:10.216ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
52282025-12-16T03:02:10.216ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
52292025-12-16T03:02:10.216ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
52302025-12-16T03:02:10.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52312025-12-16T03:02:10.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52322025-12-16T03:02:10.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52332025-12-16T03:02:10.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52342025-12-16T03:02:10.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52352025-12-16T03:02:10.222ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
52362025-12-16T03:02:10.257ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
52372025-12-16T03:02:10.258ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
52382025-12-16T03:02:10.258ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
52392025-12-16T03:02:10.258ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52402025-12-16T03:02:10.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52412025-12-16T03:02:10.259ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52422025-12-16T03:02:10.262ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
52432025-12-16T03:02:10.262ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52442025-12-16T03:02:10.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52452025-12-16T03:02:10.301ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
52462025-12-16T03:02:10.301ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
52472025-12-16T03:02:10.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52482025-12-16T03:02:10.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52492025-12-16T03:02:10.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52502025-12-16T03:02:10.307ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
52512025-12-16T03:02:10.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52522025-12-16T03:02:10.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52532025-12-16T03:02:10.309ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
52542025-12-16T03:02:10.309ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
52552025-12-16T03:02:10.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52562025-12-16T03:02:10.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52572025-12-16T03:02:10.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52582025-12-16T03:02:10.312ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
52592025-12-16T03:02:10.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52602025-12-16T03:02:10.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52612025-12-16T03:02:10.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52622025-12-16T03:02:10.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52632025-12-16T03:02:10.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52642025-12-16T03:02:10.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52652025-12-16T03:02:10.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52662025-12-16T03:02:10.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
52672025-12-16T03:02:10.327ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
52682025-12-16T03:02:10.327ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
52692025-12-16T03:02:10.327ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 95d92d50-37d5-447b-9982-09d309405007 step component = OmicronZones step id = Cleanup
52702025-12-16T03:02:10.327ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 95d92d50-37d5-447b-9982-09d309405007 step component = OmicronZones step id = Cleanup
52712025-12-16T03:02:10.327ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 95d92d50-37d5-447b-9982-09d309405007 step component = PhysicalDisks step id = Cleanup
52722025-12-16T03:02:10.327ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 95d92d50-37d5-447b-9982-09d309405007 step component = Clickhouse step id = Ensure
52732025-12-16T03:02:10.327ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 95d92d50-37d5-447b-9982-09d309405007 step component = Clickhouse step id = Ensure
52742025-12-16T03:02:10.359ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
52752025-12-16T03:02:10.364ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "a80995b3-51e8-4cd9-977c-ee5fd7bde2c2", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
52762025-12-16T03:02:10.367ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 95d92d50-37d5-447b-9982-09d309405007 step component = SupportBundles step id = Cleanup
52772025-12-16T03:02:10.367ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52782025-12-16T03:02:10.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52792025-12-16T03:02:10.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52802025-12-16T03:02:10.370ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
52812025-12-16T03:02:10.370ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52822025-12-16T03:02:10.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52832025-12-16T03:02:10.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52842025-12-16T03:02:10.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52852025-12-16T03:02:10.373ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52862025-12-16T03:02:10.376ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
52872025-12-16T03:02:10.415ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 95d92d50-37d5-447b-9982-09d309405007 step component = OmicronZones step id = Cleanup
52882025-12-16T03:02:10.415ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
52892025-12-16T03:02:10.415ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
52902025-12-16T03:02:10.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52912025-12-16T03:02:10.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52922025-12-16T03:02:10.417ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52932025-12-16T03:02:10.418ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
52942025-12-16T03:02:10.418ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52952025-12-16T03:02:10.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52962025-12-16T03:02:10.457ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
52972025-12-16T03:02:10.457ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(a91734db-9ac7-4793-9c32-ddbb21f8d986 (blueprint), false)
52982025-12-16T03:02:10.457ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 95d92d50-37d5-447b-9982-09d309405007 step component = Cockroach step id = Ensure
52992025-12-16T03:02:10.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
53002025-12-16T03:02:10.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
53012025-12-16T03:02:10.474ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
53022025-12-16T03:02:10.474ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 95d92d50-37d5-447b-9982-09d309405007 step component = MgsUpdates step id = Ensure
53032025-12-16T03:02:10.474ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 11 reason = Dependency
53042025-12-16T03:02:10.474ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 5 reason = Dependency
53052025-12-16T03:02:10.474ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
53062025-12-16T03:02:10.475ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
53072025-12-16T03:02:10.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
53082025-12-16T03:02:10.475ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.3062633s iteration = 5 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"e1244754-4dad-483c-9a27-844cc31a0ebd","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"e1244754-4dad-483c-9a27-844cc31a0ebd","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":5280,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":122396020,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":122395980,"secs":0}},"event_index":1,"execution_id":"e1244754-4dad-483c-9a27-844cc31a0ebd","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":122418931,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":250496414,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":250496344,"secs":0}},"event_index":2,"execution_id":"e1244754-4dad-483c-9a27-844cc31a0ebd","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":372932715,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":82107430,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":82107400,"secs":0}},"event_index":3,"execution_id":"e1244754-4dad-483c-9a27-844cc31a0ebd","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":455065415,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":90394256,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":90394216,"secs":0}},"event_index":4,"execution_id":"e1244754-4dad-483c-9a27-844cc31a0ebd","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":545473381,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":425962217,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":425962187,"secs":0}},"event_index":5,"execution_id":"e1244754-4dad-483c-9a27-844cc31a0ebd","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":971446648,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":187351084,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":187351044,"secs":0}},"event_index":6,"execution_id":"e1244754-4dad-483c-9a27-844cc31a0ebd","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":158805332,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3990,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":3950,"secs":0}},"event_index":7,"execution_id":"e1244754-4dad-483c-9a27-844cc31a0ebd","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":158825072,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2420,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2391,"secs":0}},"event_index":8,"execution_id":"e1244754-4dad-483c-9a27-844cc31a0ebd","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":158830013,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3389,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3349,"secs":0}},"event_index":9,"execution_id":"e1244754-4dad-483c-9a27-844cc31a0ebd","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":158835173,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1791,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1751,"secs":0}},"event_index":10,"execution_id":"e1244754-4dad-483c-9a27-844cc31a0ebd","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":158838633,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":39336318,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":39336278,"secs":0}},"event_index":11,"execution_id":"e1244754-4dad-483c-9a27-844cc31a0ebd","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":198176861,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":48101690,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":48101650,"secs":0}},"event_index":12,"execution_id":"e1244754-4dad-483c-9a27-844cc31a0ebd","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":246288411,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":41957821,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":41957781,"secs":0}},"event_index":13,"execution_id":"e1244754-4dad-483c-9a27-844cc31a0ebd","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":288266492,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":17652824,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":17652794,"secs":0}},"event_index":14,"execution_id":"e1244754-4dad-483c-9a27-844cc31a0ebd","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":305927746,"secs":1}},{"data":{"attempt_elapsed":{"nanos":6030,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5990,"secs":0}},"event_index":15,"execution_id":"e1244754-4dad-483c-9a27-844cc31a0ebd","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":305942066,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986"}
53092025-12-16T03:02:10.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
53102025-12-16T03:02:10.476ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53112025-12-16T03:02:10.476ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53122025-12-16T03:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
53132025-12-16T03:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
53142025-12-16T03:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
53152025-12-16T03:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
53162025-12-16T03:02:10.479ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
53172025-12-16T03:02:10.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
53182025-12-16T03:02:10.525ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
53192025-12-16T03:02:10.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
53202025-12-16T03:02:10.527ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53212025-12-16T03:02:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
53222025-12-16T03:02:10.529ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
53232025-12-16T03:02:10.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
53242025-12-16T03:02:10.532ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
53252025-12-16T03:02:10.532ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 57.049133ms iteration = 11 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
53262025-12-16T03:02:10.582ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(993293d3-f4d1-4544-a5c6-b0c29460233f, 0), (da536c3c-9830-4c22-a651-43d4c5674a04, 0), (cd9f1d61-49c8-4ef5-9e52-7e084cb4a60e, 0), (fecebf81-1ff4-4296-8099-25ffd5550e29, 0)] collection_id = 993293d3-f4d1-4544-a5c6-b0c29460233f
53272025-12-16T03:02:10.582ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
53282025-12-16T03:02:10.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
53292025-12-16T03:02:10.779ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 993293d3-f4d1-4544-a5c6-b0c29460233f file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
53302025-12-16T03:02:10.780ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
53312025-12-16T03:02:10.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
53322025-12-16T03:02:10.783ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(da536c3c-9830-4c22-a651-43d4c5674a04, 0), (cd9f1d61-49c8-4ef5-9e52-7e084cb4a60e, 0), (fecebf81-1ff4-4296-8099-25ffd5550e29, 0)]
53332025-12-16T03:02:10.783ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1138, tv_nsec: 399865416 } })
53342025-12-16T03:02:10.817ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("20488683-5778-4090-b36e-e6f0d80a682b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("35dbaad1-8536-44a0-8a0a-35d7c39dcfbb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b57fa226-2428-4a54-b85d-60409e28b484.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("d812a3e2-48ae-4812-a1cc-7eb7623e4e00.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::5)) }], valid_until: Instant { tv_sec: 1138, tv_nsec: 433573863 } })
53352025-12-16T03:02:10.818ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
53362025-12-16T03:02:10.818ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
53372025-12-16T03:02:10.818ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
53382025-12-16T03:02:10.819ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "181fc5ad-a7d7-4cd2-a931-06b04f27c132", "content-length": "476", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53392025-12-16T03:02:10.819ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
53402025-12-16T03:02:10.819ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e14dacad-e650-4029-9119-ebc4b16caa06", "content-length": "734", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53412025-12-16T03:02:10.819ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
53422025-12-16T03:02:10.820ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "17095d48-2632-4fae-8a0f-b97f0fd9c571", "content-length": "111", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53432025-12-16T03:02:10.820ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
53442025-12-16T03:02:10.820ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e16690e-5712-48a6-b05c-a18679e92991", "content-length": "111", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53452025-12-16T03:02:10.820ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
53462025-12-16T03:02:10.821ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d601597-24a1-4cfd-9c8a-30451da8864c", "content-length": "163", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53472025-12-16T03:02:10.821ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
53482025-12-16T03:02:10.821ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "69b5dd59-1c40-422f-987e-ba29cf47dd0e", "content-length": "163", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53492025-12-16T03:02:10.821ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
53502025-12-16T03:02:10.822ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0716b28a-321c-4fca-be51-8fa615f6a6e9", "content-length": "166", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53512025-12-16T03:02:10.822ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
53522025-12-16T03:02:10.822ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f8264cfb-251c-4be2-a71f-f380374c65f9", "content-length": "166", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53532025-12-16T03:02:10.822ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
53542025-12-16T03:02:10.823ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1010a33e-80c4-4c0a-bd95-31718dba726e", "content-length": "702", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53552025-12-16T03:02:10.823ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
53562025-12-16T03:02:10.823ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba749bcd-c870-48ca-9846-c5cee3a4f5a8", "content-length": "727", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53572025-12-16T03:02:10.823ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
53582025-12-16T03:02:10.823ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1df2f889-cd06-482f-873f-4e1de00c8cbc", "content-length": "729", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53592025-12-16T03:02:10.823ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
53602025-12-16T03:02:10.823ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "726fd2de-709c-4642-87d9-5450991e47da", "content-length": "728", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53612025-12-16T03:02:10.823ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
53622025-12-16T03:02:10.824ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dba72a9a-8115-47e5-8400-b5c39e9e7cba", "content-length": "722", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53632025-12-16T03:02:10.824ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
53642025-12-16T03:02:10.824ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bdbc970-dbbf-4baf-a1d4-5543623917c9", "content-length": "10", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53652025-12-16T03:02:10.824ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
53662025-12-16T03:02:10.825ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d10790f-d642-4694-8b97-041049ec3d5e", "content-length": "144", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53672025-12-16T03:02:10.825ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
53682025-12-16T03:02:10.825ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "aeaa3988-6e41-4ff2-b2cf-0db0a8e9f3fa", "content-length": "147", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53692025-12-16T03:02:10.825ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
53702025-12-16T03:02:10.825ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2640d52b-3e1d-4c5a-9fb4-0bf8f93217a6", "content-length": "109", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53712025-12-16T03:02:10.825ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
53722025-12-16T03:02:10.826ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cfa5933-33eb-4507-bdff-5dcc6fd320df", "content-length": "109", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53732025-12-16T03:02:10.826ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
53742025-12-16T03:02:10.826ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f9047f7-dc7b-467f-a07a-f6d242ffe2ba", "content-length": "163", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53752025-12-16T03:02:10.826ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
53762025-12-16T03:02:10.827ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "730eb937-f37b-4db1-9aa8-e2bb3f064133", "content-length": "163", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53772025-12-16T03:02:10.827ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
53782025-12-16T03:02:10.827ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bbb1b430-8da0-49cd-b756-3835dd378229", "content-length": "166", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53792025-12-16T03:02:10.828ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
53802025-12-16T03:02:10.828ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f6193a8-70fa-43fb-932a-9d54d59fd4f4", "content-length": "166", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53812025-12-16T03:02:10.828ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
53822025-12-16T03:02:10.828ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "86472963-00bc-4ea0-8bba-d37cba8f5db1", "content-length": "702", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53832025-12-16T03:02:10.828ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
53842025-12-16T03:02:10.829ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3f8dc04-c9eb-44cb-90bf-9ccdbd611aad", "content-length": "727", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53852025-12-16T03:02:10.829ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
53862025-12-16T03:02:10.829ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5193645f-4879-4dbe-a963-20672a2ccdd7", "content-length": "729", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53872025-12-16T03:02:10.829ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
53882025-12-16T03:02:10.829ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6fa170e3-3e59-493e-a7a4-2d59bfdf104b", "content-length": "728", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
53892025-12-16T03:02:10.829ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
53902025-12-16T03:02:10.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
53912025-12-16T03:02:10.831ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53922025-12-16T03:02:10.834ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
53932025-12-16T03:02:10.834ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
53942025-12-16T03:02:10.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
53952025-12-16T03:02:10.836ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53962025-12-16T03:02:10.838ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
53972025-12-16T03:02:10.838ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
53982025-12-16T03:02:10.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
53992025-12-16T03:02:10.871ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
54002025-12-16T03:02:10.871ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
54012025-12-16T03:02:10.935ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "579fa145-c9cf-474e-9817-a7e83d2cfccc", "content-length": "95612", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
54022025-12-16T03:02:10.937ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
54032025-12-16T03:02:10.937ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
54042025-12-16T03:02:10.937ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
54052025-12-16T03:02:10.937ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
54062025-12-16T03:02:10.937ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
54072025-12-16T03:02:10.937ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
54082025-12-16T03:02:10.937ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
54092025-12-16T03:02:10.937ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
54102025-12-16T03:02:10.937ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e71e239-02b4-4506-b34c-a4de2ba89987", "content-length": "64", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
54112025-12-16T03:02:10.937ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a5f6030-7aad-4f55-8277-de8b6b5ceacb", "content-length": "64", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
54122025-12-16T03:02:10.937ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
54132025-12-16T03:02:10.937ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
54142025-12-16T03:02:10.937ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd8ec799-38de-4930-bbca-333f4e08609b", "content-length": "64", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
54152025-12-16T03:02:10.937ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
54162025-12-16T03:02:10.937ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8fb70d27-f0d9-4ed2-9749-9ce494226f76", "content-length": "64", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
54172025-12-16T03:02:10.937ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
54182025-12-16T03:02:10.937ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a08f41a-b866-4ed5-8717-b84d2c7ee9f9", "content-length": "64", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
54192025-12-16T03:02:10.937ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
54202025-12-16T03:02:10.973ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "01cc80e3-a5d5-469f-b073-72d3df74360b", "content-length": "388577", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
54212025-12-16T03:02:10.974ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "14e4d3fc-2d8a-4fbb-9420-1ca2d20d2124", "content-length": "392771", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
54222025-12-16T03:02:10.975ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "72154011-3e38-4c6a-87b0-1a172326850b", "content-length": "388426", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
54232025-12-16T03:02:10.975ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d5b7585-adca-4ff5-9a72-f257ef513e8e", "content-length": "390194", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
54242025-12-16T03:02:10.980ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "953cec48-cd4e-43cf-876d-345d923ed6ed", "content-length": "421410", "date": "Tue, 16 Dec 2025 03:02:10 GMT"} })
54252025-12-16T03:02:10.983ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
54262025-12-16T03:02:10.983ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
54272025-12-16T03:02:10.983ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
54282025-12-16T03:02:10.983ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
54292025-12-16T03:02:10.983ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
54302025-12-16T03:02:11.013ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c (omicron_zone)
54312025-12-16T03:02:11.013ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
54322025-12-16T03:02:11.016ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "f40bbad7-7233-4a02-b169-23ed71bf61ab", "content-length": "125", "date": "Tue, 16 Dec 2025 03:02:11 GMT"} })
54332025-12-16T03:02:11.017ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
54342025-12-16T03:02:11.061ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 294 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 0c5f7bc0-17d6-411d-a950-0377fe44d9b1 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
54352025-12-16T03:02:11.110ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
54362025-12-16T03:02:11.110ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
54372025-12-16T03:02:11.111ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "578dc49b-9623-4916-9435-a098919c405b", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:02:11 GMT"} })
54382025-12-16T03:02:11.111ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
54392025-12-16T03:02:11.112ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
54402025-12-16T03:02:11.112ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
54412025-12-16T03:02:11.113ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "599d8baa-f145-43ae-959d-a5296fdad893", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:02:11 GMT"} })
54422025-12-16T03:02:11.113ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
54432025-12-16T03:02:11.114ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
54442025-12-16T03:02:11.114ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
54452025-12-16T03:02:11.115ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0e44a69-6862-4d1c-b2b8-a1d5b6c47643", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:02:11 GMT"} })
54462025-12-16T03:02:11.115ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
54472025-12-16T03:02:11.120ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
54482025-12-16T03:02:11.120ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
54492025-12-16T03:02:11.122ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
54502025-12-16T03:02:11.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
54512025-12-16T03:02:11.124ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54522025-12-16T03:02:11.126ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
54532025-12-16T03:02:11.126ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
54542025-12-16T03:02:11.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
54552025-12-16T03:02:11.336ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = cd59d5cd-cde1-4196-b850-0ce71d63843c file = nexus/db-queries/src/db/datastore/inventory.rs:1613
54562025-12-16T03:02:11.336ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = cd59d5cd-cde1-4196-b850-0ce71d63843c time_started = 2025-12-16 03:02:10.818542 UTC
54572025-12-16T03:02:11.336ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 14 reason = Dependency
54582025-12-16T03:02:11.336ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 18 reason = Dependency
54592025-12-16T03:02:11.336ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
54602025-12-16T03:02:11.336ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 21.78\u{b5}s iteration = 14 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
54612025-12-16T03:02:11.336ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
54622025-12-16T03:02:11.336ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 861.473115ms iteration = 5 status = {"collection_id":"cd59d5cd-cde1-4196-b850-0ce71d63843c","time_done":"2025-12-16 03:02:11.120420 UTC","time_started":"2025-12-16 03:02:10.818542 UTC"}
54632025-12-16T03:02:11.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
54642025-12-16T03:02:11.338ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54652025-12-16T03:02:11.341ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
54662025-12-16T03:02:11.341ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
54672025-12-16T03:02:11.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
54682025-12-16T03:02:11.344ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
54692025-12-16T03:02:11.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
54702025-12-16T03:02:11.392ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 15 reason = Dependency
54712025-12-16T03:02:11.392ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 55.894708ms iteration = 18 status = {"Loaded":{"collection_id":"cd59d5cd-cde1-4196-b850-0ce71d63843c","time_loaded":"2025-12-16T03:02:11.336447276Z","time_started":"2025-12-16T03:02:10.818542Z"}}
54722025-12-16T03:02:11.392ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
54732025-12-16T03:02:11.392ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 12 reason = Dependency
54742025-12-16T03:02:11.392ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 13 reason = Dependency
54752025-12-16T03:02:11.392ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 7 reason = Dependency
54762025-12-16T03:02:11.392ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 21.28\u{b5}s iteration = 15 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
54772025-12-16T03:02:11.392ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
54782025-12-16T03:02:11.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54792025-12-16T03:02:11.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
54802025-12-16T03:02:11.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
54812025-12-16T03:02:11.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
54822025-12-16T03:02:11.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54832025-12-16T03:02:11.393ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54842025-12-16T03:02:11.394ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54852025-12-16T03:02:11.394ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54862025-12-16T03:02:11.397ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
54872025-12-16T03:02:11.397ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
54882025-12-16T03:02:11.397ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
54892025-12-16T03:02:11.397ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
54902025-12-16T03:02:11.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
54912025-12-16T03:02:11.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
54922025-12-16T03:02:11.398ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
54932025-12-16T03:02:11.398ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54942025-12-16T03:02:11.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54952025-12-16T03:02:11.399ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54962025-12-16T03:02:11.399ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54972025-12-16T03:02:11.400ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
54982025-12-16T03:02:11.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
54992025-12-16T03:02:11.401ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55002025-12-16T03:02:11.402ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
55012025-12-16T03:02:11.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55022025-12-16T03:02:11.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55032025-12-16T03:02:11.402ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
55042025-12-16T03:02:11.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
55052025-12-16T03:02:11.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
55062025-12-16T03:02:11.404ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
55072025-12-16T03:02:11.404ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
55082025-12-16T03:02:11.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
55092025-12-16T03:02:11.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
55102025-12-16T03:02:11.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
55112025-12-16T03:02:11.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55122025-12-16T03:02:11.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55132025-12-16T03:02:11.406ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55142025-12-16T03:02:11.406ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55152025-12-16T03:02:11.409ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
55162025-12-16T03:02:11.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
55172025-12-16T03:02:11.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
55182025-12-16T03:02:11.409ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
55192025-12-16T03:02:11.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55202025-12-16T03:02:11.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55212025-12-16T03:02:11.410ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55222025-12-16T03:02:11.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55232025-12-16T03:02:11.413ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
55242025-12-16T03:02:11.413ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
55252025-12-16T03:02:11.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
55262025-12-16T03:02:11.413ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
55272025-12-16T03:02:11.413ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55282025-12-16T03:02:11.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55292025-12-16T03:02:11.415ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
55302025-12-16T03:02:11.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
55312025-12-16T03:02:11.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
55322025-12-16T03:02:11.417ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55332025-12-16T03:02:11.419ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
55342025-12-16T03:02:11.419ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
55352025-12-16T03:02:11.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
55362025-12-16T03:02:11.420ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55372025-12-16T03:02:11.423ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
55382025-12-16T03:02:11.423ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
55392025-12-16T03:02:11.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
55402025-12-16T03:02:11.425ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
55412025-12-16T03:02:11.425ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 33.058679ms iteration = 7 status = {"blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","inventory_collection_id":"cd59d5cd-cde1-4196-b850-0ce71d63843c","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
55422025-12-16T03:02:11.456ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 63.901229ms iteration = 13 status = {"physical_disks_added":0}
55432025-12-16T03:02:11.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55442025-12-16T03:02:11.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55452025-12-16T03:02:11.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55462025-12-16T03:02:11.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55472025-12-16T03:02:11.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55482025-12-16T03:02:11.467ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
55492025-12-16T03:02:11.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55502025-12-16T03:02:11.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55512025-12-16T03:02:11.469ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55522025-12-16T03:02:11.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55532025-12-16T03:02:11.470ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55542025-12-16T03:02:11.474ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
55552025-12-16T03:02:11.474ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55562025-12-16T03:02:11.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55572025-12-16T03:02:11.475ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55582025-12-16T03:02:11.480ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
55592025-12-16T03:02:11.480ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55602025-12-16T03:02:11.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55612025-12-16T03:02:11.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55622025-12-16T03:02:11.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55632025-12-16T03:02:11.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55642025-12-16T03:02:11.487ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
55652025-12-16T03:02:11.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55662025-12-16T03:02:11.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55672025-12-16T03:02:11.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55682025-12-16T03:02:11.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55692025-12-16T03:02:11.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55702025-12-16T03:02:11.494ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
55712025-12-16T03:02:11.494ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55722025-12-16T03:02:11.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55732025-12-16T03:02:11.496ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55742025-12-16T03:02:11.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55752025-12-16T03:02:11.497ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55762025-12-16T03:02:11.500ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
55772025-12-16T03:02:11.500ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55782025-12-16T03:02:11.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55792025-12-16T03:02:11.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55802025-12-16T03:02:11.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55812025-12-16T03:02:11.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55822025-12-16T03:02:11.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55832025-12-16T03:02:11.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55842025-12-16T03:02:11.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
55852025-12-16T03:02:11.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55862025-12-16T03:02:11.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55872025-12-16T03:02:11.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55882025-12-16T03:02:11.531ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
55892025-12-16T03:02:11.531ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55902025-12-16T03:02:11.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55912025-12-16T03:02:11.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55922025-12-16T03:02:11.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55932025-12-16T03:02:11.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55942025-12-16T03:02:11.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55952025-12-16T03:02:11.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55962025-12-16T03:02:11.540ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
55972025-12-16T03:02:11.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55982025-12-16T03:02:11.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55992025-12-16T03:02:11.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56002025-12-16T03:02:11.545ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
56012025-12-16T03:02:11.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56022025-12-16T03:02:11.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56032025-12-16T03:02:11.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56042025-12-16T03:02:11.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56052025-12-16T03:02:11.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56062025-12-16T03:02:11.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56072025-12-16T03:02:11.550ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56082025-12-16T03:02:11.553ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
56092025-12-16T03:02:11.592ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56102025-12-16T03:02:11.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56112025-12-16T03:02:11.592ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56122025-12-16T03:02:11.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56132025-12-16T03:02:11.594ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56142025-12-16T03:02:11.598ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
56152025-12-16T03:02:11.599ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56162025-12-16T03:02:11.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56172025-12-16T03:02:11.601ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56182025-12-16T03:02:11.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56192025-12-16T03:02:11.603ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56202025-12-16T03:02:11.606ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
56212025-12-16T03:02:11.606ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56222025-12-16T03:02:11.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56232025-12-16T03:02:11.608ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56242025-12-16T03:02:11.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56252025-12-16T03:02:11.610ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56262025-12-16T03:02:11.613ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
56272025-12-16T03:02:11.613ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56282025-12-16T03:02:11.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56292025-12-16T03:02:11.615ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56302025-12-16T03:02:11.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56312025-12-16T03:02:11.616ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56322025-12-16T03:02:11.619ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
56332025-12-16T03:02:11.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56342025-12-16T03:02:11.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56352025-12-16T03:02:11.621ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56362025-12-16T03:02:11.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56372025-12-16T03:02:11.622ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56382025-12-16T03:02:11.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
56392025-12-16T03:02:11.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56402025-12-16T03:02:11.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56412025-12-16T03:02:11.626ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
56422025-12-16T03:02:11.626ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
56432025-12-16T03:02:11.626ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = a22223e0-ce88-4f00-954c-20b2d9608a51 parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
56442025-12-16T03:02:11.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56452025-12-16T03:02:11.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56462025-12-16T03:02:11.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56472025-12-16T03:02:11.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
56482025-12-16T03:02:11.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
56492025-12-16T03:02:11.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
56502025-12-16T03:02:11.633ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
56512025-12-16T03:02:11.633ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
56522025-12-16T03:02:11.633ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 241.141915ms iteration = 12 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
56532025-12-16T03:02:14.209ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 38 reason = Timeout
56542025-12-16T03:02:14.209ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
56552025-12-16T03:02:14.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
56562025-12-16T03:02:14.212ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56572025-12-16T03:02:14.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
56582025-12-16T03:02:14.215ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
56592025-12-16T03:02:14.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
56602025-12-16T03:02:14.218ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
56612025-12-16T03:02:14.218ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.426097ms iteration = 38 status = {"config_updated":false}
56622025-12-16T03:02:19.209ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 20 reason = Timeout
56632025-12-16T03:02:19.209ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 39 reason = Timeout
56642025-12-16T03:02:19.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56652025-12-16T03:02:19.209ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
56662025-12-16T03:02:19.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
56672025-12-16T03:02:19.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
56682025-12-16T03:02:19.211ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 20 reason = Timeout
56692025-12-16T03:02:19.211ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
56702025-12-16T03:02:19.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
56712025-12-16T03:02:19.212ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56722025-12-16T03:02:19.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56732025-12-16T03:02:19.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
56742025-12-16T03:02:19.216ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
56752025-12-16T03:02:19.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
56762025-12-16T03:02:19.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56772025-12-16T03:02:19.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
56782025-12-16T03:02:19.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
56792025-12-16T03:02:19.218ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
56802025-12-16T03:02:19.218ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.512867ms iteration = 39 status = {"config_updated":false}
56812025-12-16T03:02:19.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56822025-12-16T03:02:19.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
56832025-12-16T03:02:19.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56842025-12-16T03:02:19.223ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
56852025-12-16T03:02:19.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56862025-12-16T03:02:19.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
56872025-12-16T03:02:19.236ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
56882025-12-16T03:02:19.236ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.144644ms iteration = 20 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
56892025-12-16T03:02:19.248ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
56902025-12-16T03:02:19.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
56912025-12-16T03:02:19.249ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 38.71185ms iteration = 20 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
56922025-12-16T03:02:21.061ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 326 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 257d91e1-3373-4060-a6c3-1ceef76fb3ee response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
56932025-12-16T03:02:24.210ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 19 reason = Timeout
56942025-12-16T03:02:24.210ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 40 reason = Timeout
56952025-12-16T03:02:24.210ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
56962025-12-16T03:02:24.210ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
56972025-12-16T03:02:24.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
56982025-12-16T03:02:24.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
56992025-12-16T03:02:24.212ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57002025-12-16T03:02:24.212ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57012025-12-16T03:02:24.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
57022025-12-16T03:02:24.216ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
57032025-12-16T03:02:24.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
57042025-12-16T03:02:24.217ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
57052025-12-16T03:02:24.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
57062025-12-16T03:02:24.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
57072025-12-16T03:02:24.218ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = cd59d5cd-cde1-4196-b850-0ce71d63843c original_time_started = 2025-12-16 03:02:10.818542 UTC
57082025-12-16T03:02:24.218ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.631567ms iteration = 19 status = {"Loaded":{"collection_id":"cd59d5cd-cde1-4196-b850-0ce71d63843c","time_loaded":"2025-12-16T03:02:24.210760950Z","time_started":"2025-12-16T03:02:10.818542Z"}}
57092025-12-16T03:02:24.219ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
57102025-12-16T03:02:24.219ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.511277ms iteration = 40 status = {"config_updated":false}
57112025-12-16T03:02:24.228ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 14 reason = Timeout
57122025-12-16T03:02:24.228ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 14 reason = Timeout
57132025-12-16T03:02:24.228ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
57142025-12-16T03:02:24.228ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
57152025-12-16T03:02:24.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
57162025-12-16T03:02:24.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
57172025-12-16T03:02:24.228ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
57182025-12-16T03:02:24.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
57192025-12-16T03:02:24.229ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57202025-12-16T03:02:24.230ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57212025-12-16T03:02:24.232ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
57222025-12-16T03:02:24.232ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
57232025-12-16T03:02:24.232ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
57242025-12-16T03:02:24.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
57252025-12-16T03:02:24.234ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.079387ms iteration = 14 status = "NoSitrep"
57262025-12-16T03:02:24.235ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
57272025-12-16T03:02:24.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
57282025-12-16T03:02:24.235ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
57292025-12-16T03:02:24.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
57302025-12-16T03:02:24.237ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57312025-12-16T03:02:24.238ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
57322025-12-16T03:02:24.239ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
57332025-12-16T03:02:24.239ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.837057ms iteration = 14 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
57342025-12-16T03:02:29.210ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 21 reason = Timeout
57352025-12-16T03:02:29.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57362025-12-16T03:02:29.210ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 41 reason = Timeout
57372025-12-16T03:02:29.210ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
57382025-12-16T03:02:29.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
57392025-12-16T03:02:29.211ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 21 reason = Timeout
57402025-12-16T03:02:29.211ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
57412025-12-16T03:02:29.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
57422025-12-16T03:02:29.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
57432025-12-16T03:02:29.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57442025-12-16T03:02:29.213ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57452025-12-16T03:02:29.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
57462025-12-16T03:02:29.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57472025-12-16T03:02:29.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
57482025-12-16T03:02:29.216ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
57492025-12-16T03:02:29.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
57502025-12-16T03:02:29.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
57512025-12-16T03:02:29.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57522025-12-16T03:02:29.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
57532025-12-16T03:02:29.219ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
57542025-12-16T03:02:29.219ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.372086ms iteration = 41 status = {"config_updated":false}
57552025-12-16T03:02:29.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57562025-12-16T03:02:29.224ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
57572025-12-16T03:02:29.224ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57582025-12-16T03:02:29.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
57592025-12-16T03:02:29.237ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
57602025-12-16T03:02:29.237ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.907181ms iteration = 21 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
57612025-12-16T03:02:29.254ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
57622025-12-16T03:02:29.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
57632025-12-16T03:02:29.256ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 44.98972ms iteration = 21 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
57642025-12-16T03:02:31.061ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 435 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 2955bdb9-9745-4d40-b23d-5ad39c7bc77b response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
57652025-12-16T03:02:34.210ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 42 reason = Timeout
57662025-12-16T03:02:34.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
57672025-12-16T03:02:34.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
57682025-12-16T03:02:34.213ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57692025-12-16T03:02:34.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
57702025-12-16T03:02:34.216ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
57712025-12-16T03:02:34.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
57722025-12-16T03:02:34.218ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
57732025-12-16T03:02:34.218ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.292492ms iteration = 42 status = {"config_updated":false}
57742025-12-16T03:02:36.054ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:41727
57752025-12-16T03:02:36.054ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 738ca7ba-baa9-41bb-87e1-a2f728c3c8d7 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending version = 1.0.0
57762025-12-16T03:02:36.054ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 738ca7ba-baa9-41bb-87e1-a2f728c3c8d7 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
57772025-12-16T03:02:36.055ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 738ca7ba-baa9-41bb-87e1-a2f728c3c8d7 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
57782025-12-16T03:02:36.058ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 3502 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 738ca7ba-baa9-41bb-87e1-a2f728c3c8d7 response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending
57792025-12-16T03:02:36.058ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 5c5686fe-5436-44dc-bd87-bfbfc57c271c uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D version = 1.0.0
57802025-12-16T03:02:36.058ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 5c5686fe-5436-44dc-bd87-bfbfc57c271c roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
57812025-12-16T03:02:36.058ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 5c5686fe-5436-44dc-bd87-bfbfc57c271c resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
57822025-12-16T03:02:36.060ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 2493 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 5c5686fe-5436-44dc-bd87-bfbfc57c271c response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D
57832025-12-16T03:02:39.169ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 8 reason = Timeout
57842025-12-16T03:02:39.169ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 8 reason = Timeout
57852025-12-16T03:02:39.169ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 8 reason = Timeout
57862025-12-16T03:02:39.169ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
57872025-12-16T03:02:39.169ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
57882025-12-16T03:02:39.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
57892025-12-16T03:02:39.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
57902025-12-16T03:02:39.175ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1166, tv_nsec: 790665967 } })
57912025-12-16T03:02:39.175ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
57922025-12-16T03:02:39.176ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1166, tv_nsec: 791649880 } })
57932025-12-16T03:02:39.178ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1166, tv_nsec: 794113901 } })
57942025-12-16T03:02:39.181ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 12.64982ms iteration = 8 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
57952025-12-16T03:02:39.210ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 20 reason = Timeout
57962025-12-16T03:02:39.210ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 22 reason = Timeout
57972025-12-16T03:02:39.210ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
57982025-12-16T03:02:39.210ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 43 reason = Timeout
57992025-12-16T03:02:39.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58002025-12-16T03:02:39.210ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
58012025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
58022025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
58032025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
58042025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 8 reason = Timeout
58052025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
58062025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 12 reason = Timeout
58072025-12-16T03:02:39.211ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
58082025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 8 reason = Timeout
58092025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 8 reason = Timeout
58102025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58112025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
58122025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
58132025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 8 reason = Timeout
58142025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58152025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 14 reason = Timeout
58162025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
58172025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 8 reason = Timeout
58182025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
58192025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 8 reason = Timeout
58202025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
58212025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 12 reason = Timeout
58222025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
58232025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
58242025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 22 reason = Timeout
58252025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58262025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
58272025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
58282025-12-16T03:02:39.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58292025-12-16T03:02:39.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
58302025-12-16T03:02:39.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
58312025-12-16T03:02:39.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
58322025-12-16T03:02:39.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58332025-12-16T03:02:39.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
58342025-12-16T03:02:39.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
58352025-12-16T03:02:39.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58362025-12-16T03:02:39.213ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58372025-12-16T03:02:39.213ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58382025-12-16T03:02:39.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58392025-12-16T03:02:39.214ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58402025-12-16T03:02:39.214ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58412025-12-16T03:02:39.214ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
58422025-12-16T03:02:39.215ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58432025-12-16T03:02:39.215ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
58442025-12-16T03:02:39.215ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
58452025-12-16T03:02:39.215ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
58462025-12-16T03:02:39.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
58472025-12-16T03:02:39.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
58482025-12-16T03:02:39.216ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58492025-12-16T03:02:39.219ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
58502025-12-16T03:02:39.219ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58512025-12-16T03:02:39.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58522025-12-16T03:02:39.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
58532025-12-16T03:02:39.220ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
58542025-12-16T03:02:39.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
58552025-12-16T03:02:39.220ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
58562025-12-16T03:02:39.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
58572025-12-16T03:02:39.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
58582025-12-16T03:02:39.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
58592025-12-16T03:02:39.220ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58602025-12-16T03:02:39.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58612025-12-16T03:02:39.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
58622025-12-16T03:02:39.221ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
58632025-12-16T03:02:39.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
58642025-12-16T03:02:39.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
58652025-12-16T03:02:39.221ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58662025-12-16T03:02:39.221ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58672025-12-16T03:02:39.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
58682025-12-16T03:02:39.222ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = cd59d5cd-cde1-4196-b850-0ce71d63843c original_time_started = 2025-12-16 03:02:10.818542 UTC
58692025-12-16T03:02:39.222ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 11.963192ms iteration = 20 status = {"Loaded":{"collection_id":"cd59d5cd-cde1-4196-b850-0ce71d63843c","time_loaded":"2025-12-16T03:02:39.210790707Z","time_started":"2025-12-16T03:02:10.818542Z"}}
58702025-12-16T03:02:39.222ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
58712025-12-16T03:02:39.222ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 11.942861ms iteration = 43 status = {"config_updated":false}
58722025-12-16T03:02:39.222ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
58732025-12-16T03:02:39.222ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
58742025-12-16T03:02:39.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
58752025-12-16T03:02:39.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
58762025-12-16T03:02:39.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
58772025-12-16T03:02:39.224ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 13.281729ms iteration = 8 status = 0
58782025-12-16T03:02:39.224ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1166, tv_nsec: 840287366 } })
58792025-12-16T03:02:39.224ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
58802025-12-16T03:02:39.224ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58812025-12-16T03:02:39.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58822025-12-16T03:02:39.225ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
58832025-12-16T03:02:39.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58842025-12-16T03:02:39.225ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1166, tv_nsec: 841216678 } })
58852025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 8 reason = Timeout
58862025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 8 reason = Timeout
58872025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 13.77\u{b5}s iteration = 8 status = {"disabled":true,"errors":[],"sps":[]}
58882025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 15 reason = Timeout
58892025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
58902025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
58912025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 15 reason = Timeout
58922025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
58932025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 8 reason = Timeout
58942025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
58952025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 8 reason = Timeout
58962025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
58972025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 8 reason = Timeout
58982025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
58992025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 8 reason = Timeout
59002025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
59012025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
59022025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
59032025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
59042025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
59052025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
59062025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
59072025-12-16T03:02:39.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
59082025-12-16T03:02:39.229ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
59092025-12-16T03:02:39.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
59102025-12-16T03:02:39.230ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
59112025-12-16T03:02:39.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
59122025-12-16T03:02:39.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
59132025-12-16T03:02:39.233ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59142025-12-16T03:02:39.236ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
59152025-12-16T03:02:39.239ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
59162025-12-16T03:02:39.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
59172025-12-16T03:02:39.240ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
59182025-12-16T03:02:39.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
59192025-12-16T03:02:39.272ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
59202025-12-16T03:02:39.272ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 61.761473ms iteration = 22 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
59212025-12-16T03:02:39.276ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
59222025-12-16T03:02:39.276ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
59232025-12-16T03:02:39.277ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "88304a47-a4f7-4c31-9e5a-1bd42442c613", "content-length": "26", "date": "Tue, 16 Dec 2025 03:02:39 GMT"} })
59242025-12-16T03:02:39.277ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
59252025-12-16T03:02:39.279ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59262025-12-16T03:02:39.281ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
59272025-12-16T03:02:39.281ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1166, tv_nsec: 897364450 } })
59282025-12-16T03:02:39.282ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
59292025-12-16T03:02:39.282ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
59302025-12-16T03:02:39.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
59312025-12-16T03:02:39.283ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
59322025-12-16T03:02:39.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
59332025-12-16T03:02:39.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
59342025-12-16T03:02:39.284ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 115.448423ms iteration = 8 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
59352025-12-16T03:02:39.284ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
59362025-12-16T03:02:39.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
59372025-12-16T03:02:39.287ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 75.645758ms iteration = 8 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
59382025-12-16T03:02:39.292ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59392025-12-16T03:02:39.293ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
59402025-12-16T03:02:39.293ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
59412025-12-16T03:02:39.293ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
59422025-12-16T03:02:39.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
59432025-12-16T03:02:39.294ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 66.449172ms iteration = 8 status = {"errors":[],"finish_invoked_ok":[]}
59442025-12-16T03:02:39.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
59452025-12-16T03:02:39.296ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
59462025-12-16T03:02:39.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
59472025-12-16T03:02:39.297ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59482025-12-16T03:02:39.301ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
59492025-12-16T03:02:39.301ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
59502025-12-16T03:02:39.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
59512025-12-16T03:02:39.302ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
59522025-12-16T03:02:39.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
59532025-12-16T03:02:39.303ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 75.603938ms iteration = 15 status = "NoSitrep"
59542025-12-16T03:02:39.305ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 77.581803ms iteration = 8 status = {"errors":[],"garbage_collect_requested":[]}
59552025-12-16T03:02:39.307ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
59562025-12-16T03:02:39.307ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 95.980606ms iteration = 8 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
59572025-12-16T03:02:39.307ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59582025-12-16T03:02:39.309ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
59592025-12-16T03:02:39.310ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
59602025-12-16T03:02:39.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
59612025-12-16T03:02:39.310ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
59622025-12-16T03:02:39.310ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 82.737008ms iteration = 15 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
59632025-12-16T03:02:39.313ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59642025-12-16T03:02:39.315ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 87.62665ms iteration = 8 status = {"errors":[],"requests_created_ok":[]}
59652025-12-16T03:02:39.316ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
59662025-12-16T03:02:39.316ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
59672025-12-16T03:02:39.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
59682025-12-16T03:02:39.318ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
59692025-12-16T03:02:39.318ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
59702025-12-16T03:02:39.318ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "21db352b-f234-44ea-a0b0-4a61f240c066", "content-length": "26", "date": "Tue, 16 Dec 2025 03:02:39 GMT"} })
59712025-12-16T03:02:39.318ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
59722025-12-16T03:02:39.319ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
59732025-12-16T03:02:39.321ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
59742025-12-16T03:02:39.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
59752025-12-16T03:02:39.322ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 110.622041ms iteration = 14 status = {"physical_disks_added":0}
59762025-12-16T03:02:39.344ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
59772025-12-16T03:02:39.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
59782025-12-16T03:02:39.344ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 133.069976ms iteration = 22 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
59792025-12-16T03:02:39.347ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59802025-12-16T03:02:39.349ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
59812025-12-16T03:02:39.349ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
59822025-12-16T03:02:39.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
59832025-12-16T03:02:39.353ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 184.173844ms iteration = 8 status = {}
59842025-12-16T03:02:39.356ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 128.325066ms iteration = 8 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
59852025-12-16T03:02:39.358ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
59862025-12-16T03:02:39.358ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
59872025-12-16T03:02:39.358ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 146.884761ms iteration = 8 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
59882025-12-16T03:02:39.363ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
59892025-12-16T03:02:39.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
59902025-12-16T03:02:39.382ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1166, tv_nsec: 998167217 } })
59912025-12-16T03:02:39.393ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
59922025-12-16T03:02:39.393ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 182.377431ms iteration = 12 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
59932025-12-16T03:02:39.403ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 175.775067ms iteration = 8 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
59942025-12-16T03:02:39.406ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
59952025-12-16T03:02:39.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
59962025-12-16T03:02:39.415ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
59972025-12-16T03:02:39.415ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
59982025-12-16T03:02:39.416ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "831d6e69-97cb-4805-bdb1-24bb2d6c6572", "content-length": "372", "date": "Tue, 16 Dec 2025 03:02:39 GMT"} })
59992025-12-16T03:02:39.416ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
60002025-12-16T03:02:39.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60012025-12-16T03:02:39.432ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1167, tv_nsec: 48091019 } })
60022025-12-16T03:02:39.460ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
60032025-12-16T03:02:39.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
60042025-12-16T03:02:39.462ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60052025-12-16T03:02:39.465ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
60062025-12-16T03:02:39.465ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
60072025-12-16T03:02:39.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
60082025-12-16T03:02:39.466ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60092025-12-16T03:02:39.466ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60102025-12-16T03:02:39.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60112025-12-16T03:02:39.469ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
60122025-12-16T03:02:39.469ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
60132025-12-16T03:02:39.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
60142025-12-16T03:02:39.469ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
60152025-12-16T03:02:39.469ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60162025-12-16T03:02:39.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60172025-12-16T03:02:39.476ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
60182025-12-16T03:02:39.476ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
60192025-12-16T03:02:39.476ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "4251bfdf-be9c-4dea-9057-8c9c3f22aecf", "date": "Tue, 16 Dec 2025 03:02:39 GMT"} })
60202025-12-16T03:02:39.476ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
60212025-12-16T03:02:39.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60222025-12-16T03:02:39.481ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
60232025-12-16T03:02:39.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60242025-12-16T03:02:39.484ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
60252025-12-16T03:02:39.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
60262025-12-16T03:02:39.486ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
60272025-12-16T03:02:39.486ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
60282025-12-16T03:02:39.487ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
60292025-12-16T03:02:39.487ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
60302025-12-16T03:02:39.487ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
60312025-12-16T03:02:39.487ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
60322025-12-16T03:02:39.487ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
60332025-12-16T03:02:39.487ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
60342025-12-16T03:02:39.487ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
60352025-12-16T03:02:39.487ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
60362025-12-16T03:02:39.488ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
60372025-12-16T03:02:39.488ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60382025-12-16T03:02:39.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60392025-12-16T03:02:39.488ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
60402025-12-16T03:02:39.488ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
60412025-12-16T03:02:39.490ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "32d3e522-5ae7-46c6-a3f9-f475411e9003", "date": "Tue, 16 Dec 2025 03:02:39 GMT"} })
60422025-12-16T03:02:39.491ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
60432025-12-16T03:02:39.491ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
60442025-12-16T03:02:39.491ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 280.304621ms iteration = 12 status = {}
60452025-12-16T03:02:39.587ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
60462025-12-16T03:02:39.588ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2b5debf-069b-40c1-9e17-69250023baa7", "content-length": "691", "date": "Tue, 16 Dec 2025 03:02:39 GMT"} })
60472025-12-16T03:02:39.588ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
60482025-12-16T03:02:39.588ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
60492025-12-16T03:02:39.588ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
60502025-12-16T03:02:39.589ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 378.10218ms iteration = 8 status = {}
60512025-12-16T03:02:39.690ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "c52730c4-22b8-4a8d-a2fb-f8ff9bcce1f4", "date": "Tue, 16 Dec 2025 03:02:39 GMT"} })
60522025-12-16T03:02:39.690ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
60532025-12-16T03:02:39.690ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
60542025-12-16T03:02:39.690ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "30c53224-cb72-43b6-a7fa-7182f4629e34", "content-length": "682", "date": "Tue, 16 Dec 2025 03:02:39 GMT"} })
60552025-12-16T03:02:39.690ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60562025-12-16T03:02:39.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60572025-12-16T03:02:39.694ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60582025-12-16T03:02:39.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60592025-12-16T03:02:39.697ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60602025-12-16T03:02:39.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60612025-12-16T03:02:39.699ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
60622025-12-16T03:02:39.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
60632025-12-16T03:02:39.706ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 494.495825ms iteration = 8 status = {}
60642025-12-16T03:02:39.913ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 1
60652025-12-16T03:02:39.913ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221
60662025-12-16T03:02:39.943ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
60672025-12-16T03:02:39.943ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:59001
60682025-12-16T03:02:39.943ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:59001 req_id = 399f8289-417b-49c6-b124-61bee7c83765 uri = /metrics/producers version = 1.0.0
60692025-12-16T03:02:39.943ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:59001 req_id = 399f8289-417b-49c6-b124-61bee7c83765 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
60702025-12-16T03:02:39.944ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:59001 req_id = 399f8289-417b-49c6-b124-61bee7c83765 resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
60712025-12-16T03:02:39.990ZINFOnexus: assigned collector to new producer collector_id = e5b81e4f-90b9-43ba-ba56-584fd121e601 file = nexus/src/app/oximeter.rs:76 producer_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
60722025-12-16T03:02:39.990ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 46271 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:59001 req_id = 399f8289-417b-49c6-b124-61bee7c83765 response_code = 201 uri = /metrics/producers
60732025-12-16T03:02:39.990ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "399f8289-417b-49c6-b124-61bee7c83765", "content-length": "41", "date": "Tue, 16 Dec 2025 03:02:39 GMT"} })
60742025-12-16T03:02:39.992ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
60752025-12-16T03:02:41.061ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 493 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = b74e4ec7-f04e-4f3a-a1ba-125362b39bc5 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
60762025-12-16T03:02:44.211ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 44 reason = Timeout
60772025-12-16T03:02:44.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
60782025-12-16T03:02:44.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
60792025-12-16T03:02:44.214ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60802025-12-16T03:02:44.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
60812025-12-16T03:02:44.217ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
60822025-12-16T03:02:44.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
60832025-12-16T03:02:44.219ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
60842025-12-16T03:02:44.219ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.66691ms iteration = 44 status = {"config_updated":false}
60852025-12-16T03:02:49.210ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 23 reason = Timeout
60862025-12-16T03:02:49.210ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 45 reason = Timeout
60872025-12-16T03:02:49.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60882025-12-16T03:02:49.210ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
60892025-12-16T03:02:49.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
60902025-12-16T03:02:49.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
60912025-12-16T03:02:49.211ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 23 reason = Timeout
60922025-12-16T03:02:49.211ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
60932025-12-16T03:02:49.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
60942025-12-16T03:02:49.212ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60952025-12-16T03:02:49.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60962025-12-16T03:02:49.213ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
60972025-12-16T03:02:49.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
60982025-12-16T03:02:49.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
60992025-12-16T03:02:49.216ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
61002025-12-16T03:02:49.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
61012025-12-16T03:02:49.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61022025-12-16T03:02:49.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
61032025-12-16T03:02:49.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
61042025-12-16T03:02:49.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61052025-12-16T03:02:49.218ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
61062025-12-16T03:02:49.218ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.395717ms iteration = 45 status = {"config_updated":false}
61072025-12-16T03:02:49.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
61082025-12-16T03:02:49.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61092025-12-16T03:02:49.223ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
61102025-12-16T03:02:49.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61112025-12-16T03:02:49.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
61122025-12-16T03:02:49.240ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
61132025-12-16T03:02:49.240ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.879279ms iteration = 23 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
61142025-12-16T03:02:49.250ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 39.390199ms iteration = 23 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
61152025-12-16T03:02:51.061ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 270 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = bc228262-944e-4a2a-9297-f829e4495354 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
61162025-12-16T03:02:54.210ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 21 reason = Timeout
61172025-12-16T03:02:54.211ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 46 reason = Timeout
61182025-12-16T03:02:54.211ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
61192025-12-16T03:02:54.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
61202025-12-16T03:02:54.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
61212025-12-16T03:02:54.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
61222025-12-16T03:02:54.213ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61232025-12-16T03:02:54.213ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61242025-12-16T03:02:54.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
61252025-12-16T03:02:54.217ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
61262025-12-16T03:02:54.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
61272025-12-16T03:02:54.217ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
61282025-12-16T03:02:54.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
61292025-12-16T03:02:54.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
61302025-12-16T03:02:54.219ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = cd59d5cd-cde1-4196-b850-0ce71d63843c original_time_started = 2025-12-16 03:02:10.818542 UTC
61312025-12-16T03:02:54.219ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.291995ms iteration = 21 status = {"Loaded":{"collection_id":"cd59d5cd-cde1-4196-b850-0ce71d63843c","time_loaded":"2025-12-16T03:02:54.210927706Z","time_started":"2025-12-16T03:02:10.818542Z"}}
61322025-12-16T03:02:54.219ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
61332025-12-16T03:02:54.219ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.564259ms iteration = 46 status = {"config_updated":false}
61342025-12-16T03:02:54.228ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 16 reason = Timeout
61352025-12-16T03:02:54.228ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 16 reason = Timeout
61362025-12-16T03:02:54.228ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
61372025-12-16T03:02:54.228ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
61382025-12-16T03:02:54.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
61392025-12-16T03:02:54.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
61402025-12-16T03:02:54.229ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
61412025-12-16T03:02:54.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
61422025-12-16T03:02:54.230ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61432025-12-16T03:02:54.230ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61442025-12-16T03:02:54.232ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
61452025-12-16T03:02:54.233ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
61462025-12-16T03:02:54.233ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
61472025-12-16T03:02:54.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
61482025-12-16T03:02:54.235ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.472562ms iteration = 16 status = "NoSitrep"
61492025-12-16T03:02:54.235ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
61502025-12-16T03:02:54.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
61512025-12-16T03:02:54.236ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
61522025-12-16T03:02:54.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
61532025-12-16T03:02:54.237ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61542025-12-16T03:02:54.238ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
61552025-12-16T03:02:54.239ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
61562025-12-16T03:02:54.239ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 11.137181ms iteration = 16 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
61572025-12-16T03:02:59.211ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 24 reason = Timeout
61582025-12-16T03:02:59.211ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 47 reason = Timeout
61592025-12-16T03:02:59.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
61602025-12-16T03:02:59.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61612025-12-16T03:02:59.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
61622025-12-16T03:02:59.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
61632025-12-16T03:02:59.211ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 24 reason = Timeout
61642025-12-16T03:02:59.211ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
61652025-12-16T03:02:59.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
61662025-12-16T03:02:59.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61672025-12-16T03:02:59.213ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61682025-12-16T03:02:59.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
61692025-12-16T03:02:59.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61702025-12-16T03:02:59.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
61712025-12-16T03:02:59.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
61722025-12-16T03:02:59.217ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
61732025-12-16T03:02:59.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
61742025-12-16T03:02:59.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61752025-12-16T03:02:59.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
61762025-12-16T03:02:59.219ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
61772025-12-16T03:02:59.219ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.018572ms iteration = 47 status = {"config_updated":false}
61782025-12-16T03:02:59.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61792025-12-16T03:02:59.222ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
61802025-12-16T03:02:59.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61812025-12-16T03:02:59.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
61822025-12-16T03:02:59.237ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
61832025-12-16T03:02:59.237ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.258192ms iteration = 24 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
61842025-12-16T03:02:59.250ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
61852025-12-16T03:02:59.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
61862025-12-16T03:02:59.294ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 82.161561ms iteration = 24 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
61872025-12-16T03:03:01.062ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 442 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 047287da-4d16-4a26-822f-9605f0e8bc64 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
61882025-12-16T03:03:04.210ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 48 reason = Timeout
61892025-12-16T03:03:04.210ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
61902025-12-16T03:03:04.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
61912025-12-16T03:03:04.213ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61922025-12-16T03:03:04.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
61932025-12-16T03:03:04.216ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
61942025-12-16T03:03:04.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
61952025-12-16T03:03:04.219ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
61962025-12-16T03:03:04.219ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.348586ms iteration = 48 status = {"config_updated":false}
61972025-12-16T03:03:06.055ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = a342d88c-3a67-4807-a124-c2ea42af47d4 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending version = 1.0.0
61982025-12-16T03:03:06.055ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = a342d88c-3a67-4807-a124-c2ea42af47d4 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
61992025-12-16T03:03:06.055ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = a342d88c-3a67-4807-a124-c2ea42af47d4 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
62002025-12-16T03:03:06.058ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 3336 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = a342d88c-3a67-4807-a124-c2ea42af47d4 response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending
62012025-12-16T03:03:06.059ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 6b162cf2-2731-4d69-a1b4-58773096d7f6 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D version = 1.0.0
62022025-12-16T03:03:06.059ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 6b162cf2-2731-4d69-a1b4-58773096d7f6 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
62032025-12-16T03:03:06.059ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 6b162cf2-2731-4d69-a1b4-58773096d7f6 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
62042025-12-16T03:03:06.062ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 3172 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 6b162cf2-2731-4d69-a1b4-58773096d7f6 response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D
62052025-12-16T03:03:09.170ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 5 reason = Timeout
62062025-12-16T03:03:09.170ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 7 reason = Timeout
62072025-12-16T03:03:09.170ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 5 reason = Timeout
62082025-12-16T03:03:09.170ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
62092025-12-16T03:03:09.170ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 5 reason = Timeout
62102025-12-16T03:03:09.170ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
62112025-12-16T03:03:09.170ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 5 reason = Timeout
62122025-12-16T03:03:09.170ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
62132025-12-16T03:03:09.170ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 5 reason = Timeout
62142025-12-16T03:03:09.170ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-12-16 02:53:09.170499491 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
62152025-12-16T03:03:09.170ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
62162025-12-16T03:03:09.170ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 7 reason = Timeout
62172025-12-16T03:03:09.170ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
62182025-12-16T03:03:09.170ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 5 reason = Timeout
62192025-12-16T03:03:09.170ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 9 reason = Timeout
62202025-12-16T03:03:09.170ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
62212025-12-16T03:03:09.170ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
62222025-12-16T03:03:09.170ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 9 reason = Timeout
62232025-12-16T03:03:09.170ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
62242025-12-16T03:03:09.170ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
62252025-12-16T03:03:09.170ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 6 reason = Timeout
62262025-12-16T03:03:09.170ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
62272025-12-16T03:03:09.170ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
62282025-12-16T03:03:09.170ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 97dbb60b-ce91-4d1d-9df6-232a11e8c982 step component = ExternalNetworking step id = Ensure
62292025-12-16T03:03:09.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
62302025-12-16T03:03:09.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
62312025-12-16T03:03:09.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
62322025-12-16T03:03:09.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
62332025-12-16T03:03:09.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
62342025-12-16T03:03:09.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
62352025-12-16T03:03:09.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
62362025-12-16T03:03:09.171ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1a45cd23-3d0a-43eb-adce-62c7b712baf1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1196, tv_nsec: 786303472 } })
62372025-12-16T03:03:09.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
62382025-12-16T03:03:09.171ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("49a51a59-9919-48ae-8182-4070a8430160.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6af60386-8771-4401-9d4d-e93774e519f0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 1196, tv_nsec: 786459194 } })
62392025-12-16T03:03:09.171ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
62402025-12-16T03:03:09.171ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.56914ms iteration = 5 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
62412025-12-16T03:03:09.172ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
62422025-12-16T03:03:09.172ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.805433ms iteration = 5 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
62432025-12-16T03:03:09.172ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
62442025-12-16T03:03:09.172ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
62452025-12-16T03:03:09.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
62462025-12-16T03:03:09.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
62472025-12-16T03:03:09.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
62482025-12-16T03:03:09.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
62492025-12-16T03:03:09.176ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62502025-12-16T03:03:09.176ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1196, tv_nsec: 791851442 } })
62512025-12-16T03:03:09.177ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1196, tv_nsec: 792971517 } })
62522025-12-16T03:03:09.178ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
62532025-12-16T03:03:09.178ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1196, tv_nsec: 793775007 } })
62542025-12-16T03:03:09.181ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
62552025-12-16T03:03:09.181ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
62562025-12-16T03:03:09.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
62572025-12-16T03:03:09.185ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
62582025-12-16T03:03:09.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
62592025-12-16T03:03:09.189ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 18.621626ms iteration = 9 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
62602025-12-16T03:03:09.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
62612025-12-16T03:03:09.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
62622025-12-16T03:03:09.192ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-12-16 02:53:09.170499491 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
62632025-12-16T03:03:09.192ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 22.162551ms iteration = 5 status = {"expiration":"2025-12-16T02:53:09.170499491Z","pruned":[]}
62642025-12-16T03:03:09.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62652025-12-16T03:03:09.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
62662025-12-16T03:03:09.210ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 13 reason = Timeout
62672025-12-16T03:03:09.210ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 25 reason = Timeout
62682025-12-16T03:03:09.210ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 22 reason = Timeout
62692025-12-16T03:03:09.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62702025-12-16T03:03:09.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62712025-12-16T03:03:09.210ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
62722025-12-16T03:03:09.210ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 49 reason = Timeout
62732025-12-16T03:03:09.210ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
62742025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
62752025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
62762025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62772025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
62782025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 5 reason = Timeout
62792025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 15 reason = Timeout
62802025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
62812025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
62822025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 13 reason = Timeout
62832025-12-16T03:03:09.211ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
62842025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 9 reason = Timeout
62852025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 9 reason = Timeout
62862025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
62872025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
62882025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
62892025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
62902025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 9 reason = Timeout
62912025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 13 reason = Timeout
62922025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
62932025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 5 reason = Timeout
62942025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 5 reason = Timeout
62952025-12-16T03:03:09.211ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
62962025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
62972025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
62982025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
62992025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 9 reason = Timeout
63002025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
63012025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 9 reason = Timeout
63022025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
63032025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 25 reason = Timeout
63042025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
63052025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 9 reason = Timeout
63062025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
63072025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
63082025-12-16T03:03:09.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
63092025-12-16T03:03:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
63102025-12-16T03:03:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
63112025-12-16T03:03:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
63122025-12-16T03:03:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
63132025-12-16T03:03:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
63142025-12-16T03:03:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
63152025-12-16T03:03:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
63162025-12-16T03:03:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
63172025-12-16T03:03:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
63182025-12-16T03:03:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
63192025-12-16T03:03:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63202025-12-16T03:03:09.224ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
63212025-12-16T03:03:09.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
63222025-12-16T03:03:09.228ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 16 reason = Timeout
63232025-12-16T03:03:09.228ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
63242025-12-16T03:03:09.228ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 9 reason = Timeout
63252025-12-16T03:03:09.228ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
63262025-12-16T03:03:09.228ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 25.97\u{b5}s iteration = 16 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
63272025-12-16T03:03:09.228ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 5 reason = Timeout
63282025-12-16T03:03:09.228ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 9 reason = Timeout
63292025-12-16T03:03:09.228ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
63302025-12-16T03:03:09.228ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
63312025-12-16T03:03:09.228ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 5 reason = Timeout
63322025-12-16T03:03:09.228ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
63332025-12-16T03:03:09.228ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
63342025-12-16T03:03:09.228ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 9 reason = Timeout
63352025-12-16T03:03:09.228ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 9 reason = Timeout
63362025-12-16T03:03:09.228ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
63372025-12-16T03:03:09.228ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 22.08\u{b5}s iteration = 9 status = {"disabled":true,"errors":[],"sps":[]}
63382025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 5 reason = Timeout
63392025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 9 reason = Timeout
63402025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
63412025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
63422025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 9 reason = Timeout
63432025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
63442025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 17 reason = Timeout
63452025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
63462025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 5 reason = Timeout
63472025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
63482025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 17 reason = Timeout
63492025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
63502025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
63512025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
63522025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
63532025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
63542025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
63552025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
63562025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
63572025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
63582025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
63592025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
63602025-12-16T03:03:09.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
63612025-12-16T03:03:09.233ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
63622025-12-16T03:03:09.236ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63632025-12-16T03:03:09.238ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
63642025-12-16T03:03:09.241ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
63652025-12-16T03:03:09.243ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
63662025-12-16T03:03:09.243ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
63672025-12-16T03:03:09.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
63682025-12-16T03:03:09.244ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
63692025-12-16T03:03:09.249ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
63702025-12-16T03:03:09.252ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
63712025-12-16T03:03:09.254ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
63722025-12-16T03:03:09.256ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
63732025-12-16T03:03:09.258ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
63742025-12-16T03:03:09.260ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
63752025-12-16T03:03:09.262ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
63762025-12-16T03:03:09.278ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
63772025-12-16T03:03:09.281ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 97dbb60b-ce91-4d1d-9df6-232a11e8c982 step component = SledList step id = Fetch
63782025-12-16T03:03:09.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
63792025-12-16T03:03:09.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
63802025-12-16T03:03:09.300ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63812025-12-16T03:03:09.302ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63822025-12-16T03:03:09.307ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
63832025-12-16T03:03:09.307ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63842025-12-16T03:03:09.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
63852025-12-16T03:03:09.308ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
63862025-12-16T03:03:09.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
63872025-12-16T03:03:09.310ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63882025-12-16T03:03:09.311ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
63892025-12-16T03:03:09.312ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
63902025-12-16T03:03:09.312ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
63912025-12-16T03:03:09.312ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
63922025-12-16T03:03:09.312ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
63932025-12-16T03:03:09.313ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63942025-12-16T03:03:09.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
63952025-12-16T03:03:09.315ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
63962025-12-16T03:03:09.316ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63972025-12-16T03:03:09.317ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63982025-12-16T03:03:09.317ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
63992025-12-16T03:03:09.317ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
64002025-12-16T03:03:09.317ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 88.616413ms iteration = 5 status = {"by_rx":{},"error":null}
64012025-12-16T03:03:09.317ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
64022025-12-16T03:03:09.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
64032025-12-16T03:03:09.318ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
64042025-12-16T03:03:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
64052025-12-16T03:03:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
64062025-12-16T03:03:09.319ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64072025-12-16T03:03:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64082025-12-16T03:03:09.319ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64092025-12-16T03:03:09.321ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
64102025-12-16T03:03:09.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
64112025-12-16T03:03:09.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
64122025-12-16T03:03:09.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64132025-12-16T03:03:09.325ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
64142025-12-16T03:03:09.325ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
64152025-12-16T03:03:09.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
64162025-12-16T03:03:09.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
64172025-12-16T03:03:09.326ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
64182025-12-16T03:03:09.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
64192025-12-16T03:03:09.328ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64202025-12-16T03:03:09.328ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
64212025-12-16T03:03:09.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
64222025-12-16T03:03:09.329ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
64232025-12-16T03:03:09.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
64242025-12-16T03:03:09.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
64252025-12-16T03:03:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
64262025-12-16T03:03:09.332ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
64272025-12-16T03:03:09.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
64282025-12-16T03:03:09.337ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 108.401024ms iteration = 5 status = {"errors":[],"found_port_ok":[]}
64292025-12-16T03:03:09.342ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = cd59d5cd-cde1-4196-b850-0ce71d63843c original_time_started = 2025-12-16 03:02:10.818542 UTC
64302025-12-16T03:03:09.342ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 131.606028ms iteration = 22 status = {"Loaded":{"collection_id":"cd59d5cd-cde1-4196-b850-0ce71d63843c","time_loaded":"2025-12-16T03:03:09.210760211Z","time_started":"2025-12-16T03:02:10.818542Z"}}
64312025-12-16T03:03:09.342ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64322025-12-16T03:03:09.343ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64332025-12-16T03:03:09.343ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64342025-12-16T03:03:09.346ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
64352025-12-16T03:03:09.346ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
64362025-12-16T03:03:09.346ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 135.179293ms iteration = 5 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
64372025-12-16T03:03:09.346ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 135.395036ms iteration = 9 status = 0
64382025-12-16T03:03:09.347ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
64392025-12-16T03:03:09.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64402025-12-16T03:03:09.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64412025-12-16T03:03:09.348ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
64422025-12-16T03:03:09.348ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
64432025-12-16T03:03:09.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
64442025-12-16T03:03:09.348ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
64452025-12-16T03:03:09.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
64462025-12-16T03:03:09.350ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 121.580851ms iteration = 17 status = "NoSitrep"
64472025-12-16T03:03:09.350ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
64482025-12-16T03:03:09.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64492025-12-16T03:03:09.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
64502025-12-16T03:03:09.352ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1196, tv_nsec: 967556469 } })
64512025-12-16T03:03:09.352ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
64522025-12-16T03:03:09.352ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
64532025-12-16T03:03:09.353ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
64542025-12-16T03:03:09.353ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
64552025-12-16T03:03:09.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
64562025-12-16T03:03:09.353ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1196, tv_nsec: 968574392 } })
64572025-12-16T03:03:09.355ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64582025-12-16T03:03:09.355ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
64592025-12-16T03:03:09.355ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64602025-12-16T03:03:09.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
64612025-12-16T03:03:09.356ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
64622025-12-16T03:03:09.356ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 145.784947ms iteration = 49 status = {"config_updated":false}
64632025-12-16T03:03:09.356ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64642025-12-16T03:03:09.356ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64652025-12-16T03:03:09.356ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64662025-12-16T03:03:09.357ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64672025-12-16T03:03:09.360ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64682025-12-16T03:03:09.360ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 131.992673ms iteration = 5 status = {"dispatched":[],"errors":[],"glob_version":"212.0.0","globs_reprocessed":{},"no_receivers":[]}
64692025-12-16T03:03:09.360ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
64702025-12-16T03:03:09.361ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
64712025-12-16T03:03:09.361ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
64722025-12-16T03:03:09.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
64732025-12-16T03:03:09.362ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
64742025-12-16T03:03:09.362ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
64752025-12-16T03:03:09.362ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
64762025-12-16T03:03:09.362ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64772025-12-16T03:03:09.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
64782025-12-16T03:03:09.362ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
64792025-12-16T03:03:09.362ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
64802025-12-16T03:03:09.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
64812025-12-16T03:03:09.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64822025-12-16T03:03:09.364ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
64832025-12-16T03:03:09.364ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
64842025-12-16T03:03:09.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
64852025-12-16T03:03:09.364ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
64862025-12-16T03:03:09.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64872025-12-16T03:03:09.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64882025-12-16T03:03:09.366ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64892025-12-16T03:03:09.371ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
64902025-12-16T03:03:09.371ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
64912025-12-16T03:03:09.371ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64922025-12-16T03:03:09.371ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
64932025-12-16T03:03:09.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
64942025-12-16T03:03:09.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64952025-12-16T03:03:09.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
64962025-12-16T03:03:09.372ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
64972025-12-16T03:03:09.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
64982025-12-16T03:03:09.379ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 97dbb60b-ce91-4d1d-9df6-232a11e8c982 step component = DeployNexusRecords step id = Ensure
64992025-12-16T03:03:09.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65002025-12-16T03:03:09.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65012025-12-16T03:03:09.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65022025-12-16T03:03:09.390ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
65032025-12-16T03:03:09.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65042025-12-16T03:03:09.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65052025-12-16T03:03:09.396ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
65062025-12-16T03:03:09.396ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
65072025-12-16T03:03:09.400ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b67fa33-5edf-41e1-bff8-e76bd9ca72a2", "content-length": "26", "date": "Tue, 16 Dec 2025 03:03:09 GMT"} })
65082025-12-16T03:03:09.401ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
65092025-12-16T03:03:09.401ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65102025-12-16T03:03:09.403ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
65112025-12-16T03:03:09.404ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1197, tv_nsec: 19421356 } })
65122025-12-16T03:03:09.406ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
65132025-12-16T03:03:09.406ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
65142025-12-16T03:03:09.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
65152025-12-16T03:03:09.408ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
65162025-12-16T03:03:09.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
65172025-12-16T03:03:09.410ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
65182025-12-16T03:03:09.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
65192025-12-16T03:03:09.412ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65202025-12-16T03:03:09.412ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 183.997821ms iteration = 9 status = {"errors":[],"garbage_collect_requested":[]}
65212025-12-16T03:03:09.414ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65222025-12-16T03:03:09.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65232025-12-16T03:03:09.415ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
65242025-12-16T03:03:09.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
65252025-12-16T03:03:09.417ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65262025-12-16T03:03:09.418ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
65272025-12-16T03:03:09.418ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
65282025-12-16T03:03:09.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
65292025-12-16T03:03:09.420ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
65302025-12-16T03:03:09.420ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 191.999243ms iteration = 9 status = {"errors":[],"finish_invoked_ok":[]}
65312025-12-16T03:03:09.421ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
65322025-12-16T03:03:09.421ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 192.742132ms iteration = 17 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
65332025-12-16T03:03:09.422ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65342025-12-16T03:03:09.428ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
65352025-12-16T03:03:09.436ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
65362025-12-16T03:03:09.436ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 225.162062ms iteration = 25 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
65372025-12-16T03:03:09.438ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
65382025-12-16T03:03:09.438ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
65392025-12-16T03:03:09.438ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
65402025-12-16T03:03:09.438ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
65412025-12-16T03:03:09.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
65422025-12-16T03:03:09.439ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 268.781495ms iteration = 9 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
65432025-12-16T03:03:09.442ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
65442025-12-16T03:03:09.444ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 215.088455ms iteration = 9 status = {"errors":[],"requests_created_ok":[]}
65452025-12-16T03:03:09.445ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65462025-12-16T03:03:09.448ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
65472025-12-16T03:03:09.448ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
65482025-12-16T03:03:09.448ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 236.834481ms iteration = 5 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
65492025-12-16T03:03:09.449ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
65502025-12-16T03:03:09.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
65512025-12-16T03:03:09.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
65522025-12-16T03:03:09.456ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
65532025-12-16T03:03:09.456ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 245.904405ms iteration = 25 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
65542025-12-16T03:03:09.462ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
65552025-12-16T03:03:09.463ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
65562025-12-16T03:03:09.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
65572025-12-16T03:03:09.466ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65582025-12-16T03:03:09.467ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
65592025-12-16T03:03:09.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
65602025-12-16T03:03:09.470ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
65612025-12-16T03:03:09.470ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
65622025-12-16T03:03:09.470ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
65632025-12-16T03:03:09.471ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
65642025-12-16T03:03:09.471ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
65652025-12-16T03:03:09.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
65662025-12-16T03:03:09.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
65672025-12-16T03:03:09.472ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65682025-12-16T03:03:09.478ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
65692025-12-16T03:03:09.478ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
65702025-12-16T03:03:09.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
65712025-12-16T03:03:09.479ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
65722025-12-16T03:03:09.479ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
65732025-12-16T03:03:09.479ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
65742025-12-16T03:03:09.479ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
65752025-12-16T03:03:09.479ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 267.918724ms iteration = 9 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
65762025-12-16T03:03:09.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
65772025-12-16T03:03:09.479ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 268.253568ms iteration = 9 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
65782025-12-16T03:03:09.483ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
65792025-12-16T03:03:09.483ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
65802025-12-16T03:03:09.483ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-12-16 02:59:16.791641 UTC dns_group = internal generation = 1
65812025-12-16T03:03:09.483ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 312.770112ms iteration = 5 status = {"generation":1}
65822025-12-16T03:03:09.502ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
65832025-12-16T03:03:09.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
65842025-12-16T03:03:09.511ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 97dbb60b-ce91-4d1d-9df6-232a11e8c982 step component = SledAgent step id = Ensure
65852025-12-16T03:03:09.515ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65862025-12-16T03:03:09.520ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 291.27915ms iteration = 9 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
65872025-12-16T03:03:09.521ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 309.744125ms iteration = 15 status = {"physical_disks_added":0}
65882025-12-16T03:03:09.521ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
65892025-12-16T03:03:09.521ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
65902025-12-16T03:03:09.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
65912025-12-16T03:03:09.539ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
65922025-12-16T03:03:09.539ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65932025-12-16T03:03:09.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
65942025-12-16T03:03:09.542ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
65952025-12-16T03:03:09.542ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
65962025-12-16T03:03:09.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
65972025-12-16T03:03:09.552ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
65982025-12-16T03:03:09.552ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 341.353445ms iteration = 13 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
65992025-12-16T03:03:09.569ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
66002025-12-16T03:03:09.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
66012025-12-16T03:03:09.571ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
66022025-12-16T03:03:09.571ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
66032025-12-16T03:03:09.571ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 359.608006ms iteration = 9 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
66042025-12-16T03:03:09.574ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 403.558003ms iteration = 9 status = {}
66052025-12-16T03:03:09.575ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66062025-12-16T03:03:09.577ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
66072025-12-16T03:03:09.577ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
66082025-12-16T03:03:09.578ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8339f10d-5e14-46de-96f4-1316aa95188f", "content-length": "372", "date": "Tue, 16 Dec 2025 03:03:09 GMT"} })
66092025-12-16T03:03:09.578ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66102025-12-16T03:03:09.578ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
66112025-12-16T03:03:09.578ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
66122025-12-16T03:03:09.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66132025-12-16T03:03:09.579ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a04b7c4-58e5-45a7-9635-107ea1856152", "content-length": "26", "date": "Tue, 16 Dec 2025 03:03:09 GMT"} })
66142025-12-16T03:03:09.579ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
66152025-12-16T03:03:09.581ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
66162025-12-16T03:03:09.581ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
66172025-12-16T03:03:09.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
66182025-12-16T03:03:09.582ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
66192025-12-16T03:03:09.588ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 359.565435ms iteration = 9 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
66202025-12-16T03:03:09.601ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
66212025-12-16T03:03:09.601ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
66222025-12-16T03:03:09.601ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
66232025-12-16T03:03:09.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
66242025-12-16T03:03:09.603ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66252025-12-16T03:03:09.604ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
66262025-12-16T03:03:09.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
66272025-12-16T03:03:09.606ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66282025-12-16T03:03:09.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66292025-12-16T03:03:09.609ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66302025-12-16T03:03:09.609ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
66312025-12-16T03:03:09.609ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
66322025-12-16T03:03:09.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
66332025-12-16T03:03:09.612ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
66342025-12-16T03:03:09.612ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
66352025-12-16T03:03:09.612ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-12-16 02:59:18.154440 UTC dns_group = external generation = 2
66362025-12-16T03:03:09.612ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 441.733117ms iteration = 5 status = {"generation":2}
66372025-12-16T03:03:09.613ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
66382025-12-16T03:03:09.613ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66392025-12-16T03:03:09.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66402025-12-16T03:03:09.615ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
66412025-12-16T03:03:09.615ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "f5c66cea-d8ca-46f9-864f-3c42f050f9c6", "date": "Tue, 16 Dec 2025 03:03:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
66422025-12-16T03:03:09.615ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
66432025-12-16T03:03:09.616ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66442025-12-16T03:03:09.619ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "0d358c53-5ca0-4e2e-8b9f-c43e10d2ef3e", "date": "Tue, 16 Dec 2025 03:03:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
66452025-12-16T03:03:09.619ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
66462025-12-16T03:03:09.621ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
66472025-12-16T03:03:09.621ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66482025-12-16T03:03:09.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66492025-12-16T03:03:09.622ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
66502025-12-16T03:03:09.622ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 410.433229ms iteration = 5 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
66512025-12-16T03:03:09.622ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
66522025-12-16T03:03:09.622ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 452.24472ms iteration = 7 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
66532025-12-16T03:03:09.622ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-12-16T02:59:05.020237Z, time_modified: 2025-12-16T02:59:05.020237Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }) }
66542025-12-16T03:03:09.622ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 452.26617ms iteration = 5 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]}},"default_endpoint":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
66552025-12-16T03:03:09.630ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
66562025-12-16T03:03:09.631ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "bc7700f1-3a18-4b29-8fc3-1e7d01d62071", "date": "Tue, 16 Dec 2025 03:03:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
66572025-12-16T03:03:09.631ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
66582025-12-16T03:03:09.634ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "4632c20a-d8eb-41bf-873e-a6d1f78610f2", "date": "Tue, 16 Dec 2025 03:03:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
66592025-12-16T03:03:09.634ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
66602025-12-16T03:03:09.638ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "7ca24b67-75c5-4c9c-a192-4ee8a12e6fdd", "date": "Tue, 16 Dec 2025 03:03:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
66612025-12-16T03:03:09.638ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
66622025-12-16T03:03:09.640ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
66632025-12-16T03:03:09.641ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
66642025-12-16T03:03:09.641ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 471.446384ms iteration = 7 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
66652025-12-16T03:03:09.642ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "f73bbb74-7585-474e-a01e-72b569addf6f", "date": "Tue, 16 Dec 2025 03:03:09 GMT"} }) sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
66662025-12-16T03:03:09.645ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 97dbb60b-ce91-4d1d-9df6-232a11e8c982 step component = FirewallRules step id = Ensure
66672025-12-16T03:03:09.645ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
66682025-12-16T03:03:09.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
66692025-12-16T03:03:09.648ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
66702025-12-16T03:03:09.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
66712025-12-16T03:03:09.651ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
66722025-12-16T03:03:09.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
66732025-12-16T03:03:09.654ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
66742025-12-16T03:03:09.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
66752025-12-16T03:03:09.659ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
66762025-12-16T03:03:09.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
66772025-12-16T03:03:09.662ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
66782025-12-16T03:03:09.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
66792025-12-16T03:03:09.664ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66802025-12-16T03:03:09.665ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
66812025-12-16T03:03:09.665ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
66822025-12-16T03:03:09.665ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f7153cf1-77bb-4657-b9bf-4b1813801d00", "date": "Tue, 16 Dec 2025 03:03:09 GMT"} })
66832025-12-16T03:03:09.665ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66842025-12-16T03:03:09.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66852025-12-16T03:03:09.668ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66862025-12-16T03:03:09.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66872025-12-16T03:03:09.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
66882025-12-16T03:03:09.671ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
66892025-12-16T03:03:09.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
66902025-12-16T03:03:09.672ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66912025-12-16T03:03:09.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66922025-12-16T03:03:09.673ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
66932025-12-16T03:03:09.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
66942025-12-16T03:03:09.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
66952025-12-16T03:03:09.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
66962025-12-16T03:03:09.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66972025-12-16T03:03:09.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66982025-12-16T03:03:09.676ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
66992025-12-16T03:03:09.676ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
67002025-12-16T03:03:09.677ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "2436fd66-53a7-4653-8654-f9a07c5ef66f", "date": "Tue, 16 Dec 2025 03:03:09 GMT"} })
67012025-12-16T03:03:09.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67022025-12-16T03:03:09.677ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
67032025-12-16T03:03:09.678ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "429f64ca-b39b-421e-9990-ef97d07b4fd0", "date": "Tue, 16 Dec 2025 03:03:09 GMT"} })
67042025-12-16T03:03:09.678ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
67052025-12-16T03:03:09.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67062025-12-16T03:03:09.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67072025-12-16T03:03:09.679ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 467.762306ms iteration = 13 status = {}
67082025-12-16T03:03:09.679ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1197, tv_nsec: 294335929 } })
67092025-12-16T03:03:09.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67102025-12-16T03:03:09.681ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
67112025-12-16T03:03:09.681ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 452.729326ms iteration = 5 status = {"errors":[],"probes_by_sled":{"b1c76b61-acee-4be6-b582-973c938d6b6e":0}}
67122025-12-16T03:03:09.681ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 14 reason = Dependency
67132025-12-16T03:03:09.681ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
67142025-12-16T03:03:09.681ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67152025-12-16T03:03:09.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67162025-12-16T03:03:09.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
67172025-12-16T03:03:09.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67182025-12-16T03:03:09.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67192025-12-16T03:03:09.683ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67202025-12-16T03:03:09.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
67212025-12-16T03:03:09.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67222025-12-16T03:03:09.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67232025-12-16T03:03:09.686ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
67242025-12-16T03:03:09.686ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67252025-12-16T03:03:09.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67262025-12-16T03:03:09.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67272025-12-16T03:03:09.690ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67282025-12-16T03:03:09.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67292025-12-16T03:03:09.692ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
67302025-12-16T03:03:09.692ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67312025-12-16T03:03:09.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67322025-12-16T03:03:09.693ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67332025-12-16T03:03:09.693ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67342025-12-16T03:03:09.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67352025-12-16T03:03:09.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67362025-12-16T03:03:09.695ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67372025-12-16T03:03:09.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67382025-12-16T03:03:09.696ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67392025-12-16T03:03:09.698ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67402025-12-16T03:03:09.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67412025-12-16T03:03:09.700ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67422025-12-16T03:03:09.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67432025-12-16T03:03:09.700ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
67442025-12-16T03:03:09.700ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67452025-12-16T03:03:09.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67462025-12-16T03:03:09.705ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67472025-12-16T03:03:09.705ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67482025-12-16T03:03:09.705ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67492025-12-16T03:03:09.708ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67502025-12-16T03:03:09.708ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
67512025-12-16T03:03:09.708ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67522025-12-16T03:03:09.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67532025-12-16T03:03:09.712ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67542025-12-16T03:03:09.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67552025-12-16T03:03:09.713ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
67562025-12-16T03:03:09.713ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67572025-12-16T03:03:09.713ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67582025-12-16T03:03:09.714ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67592025-12-16T03:03:09.716ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67602025-12-16T03:03:09.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67612025-12-16T03:03:09.718ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
67622025-12-16T03:03:09.718ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67632025-12-16T03:03:09.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67642025-12-16T03:03:09.719ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67652025-12-16T03:03:09.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67662025-12-16T03:03:09.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67672025-12-16T03:03:09.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67682025-12-16T03:03:09.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67692025-12-16T03:03:09.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67702025-12-16T03:03:09.723ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67712025-12-16T03:03:09.724ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67722025-12-16T03:03:09.724ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67732025-12-16T03:03:09.726ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67742025-12-16T03:03:09.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67752025-12-16T03:03:09.727ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
67762025-12-16T03:03:09.727ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67772025-12-16T03:03:09.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67782025-12-16T03:03:09.728ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67792025-12-16T03:03:09.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67802025-12-16T03:03:09.728ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67812025-12-16T03:03:09.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67822025-12-16T03:03:09.730ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67832025-12-16T03:03:09.730ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67842025-12-16T03:03:09.735ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
67852025-12-16T03:03:09.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67862025-12-16T03:03:09.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67872025-12-16T03:03:09.736ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1197, tv_nsec: 351730236 } })
67882025-12-16T03:03:09.739ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
67892025-12-16T03:03:09.739ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67902025-12-16T03:03:09.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67912025-12-16T03:03:09.742ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67922025-12-16T03:03:09.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67932025-12-16T03:03:09.744ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67942025-12-16T03:03:09.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67952025-12-16T03:03:09.747ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67962025-12-16T03:03:09.753ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
67972025-12-16T03:03:09.753ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67982025-12-16T03:03:09.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67992025-12-16T03:03:09.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68002025-12-16T03:03:09.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68012025-12-16T03:03:09.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68022025-12-16T03:03:09.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68032025-12-16T03:03:09.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68042025-12-16T03:03:09.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68052025-12-16T03:03:09.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68062025-12-16T03:03:09.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68072025-12-16T03:03:09.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68082025-12-16T03:03:09.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68092025-12-16T03:03:09.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68102025-12-16T03:03:09.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68112025-12-16T03:03:09.774ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68122025-12-16T03:03:09.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68132025-12-16T03:03:09.775ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68142025-12-16T03:03:09.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68152025-12-16T03:03:09.776ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68162025-12-16T03:03:09.777ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68172025-12-16T03:03:09.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68182025-12-16T03:03:09.780ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68192025-12-16T03:03:09.781ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
68202025-12-16T03:03:09.781ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68212025-12-16T03:03:09.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68222025-12-16T03:03:09.783ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68232025-12-16T03:03:09.784ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
68242025-12-16T03:03:09.784ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
68252025-12-16T03:03:09.788ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bc017b1-b5ea-4625-897f-728a7eb826a1", "content-length": "372", "date": "Tue, 16 Dec 2025 03:03:09 GMT"} })
68262025-12-16T03:03:09.789ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
68272025-12-16T03:03:09.789ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
68282025-12-16T03:03:09.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68292025-12-16T03:03:09.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
68302025-12-16T03:03:09.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68312025-12-16T03:03:09.789ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
68322025-12-16T03:03:09.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68332025-12-16T03:03:09.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68342025-12-16T03:03:09.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68352025-12-16T03:03:09.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68362025-12-16T03:03:09.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68372025-12-16T03:03:09.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68382025-12-16T03:03:09.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68392025-12-16T03:03:09.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68402025-12-16T03:03:09.797ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68412025-12-16T03:03:09.797ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68422025-12-16T03:03:09.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68432025-12-16T03:03:09.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68442025-12-16T03:03:09.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68452025-12-16T03:03:09.799ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
68462025-12-16T03:03:09.799ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68472025-12-16T03:03:09.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68482025-12-16T03:03:09.802ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
68492025-12-16T03:03:09.802ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68502025-12-16T03:03:09.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68512025-12-16T03:03:09.805ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68522025-12-16T03:03:09.809ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
68532025-12-16T03:03:09.809ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68542025-12-16T03:03:09.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68552025-12-16T03:03:09.815ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68562025-12-16T03:03:09.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68572025-12-16T03:03:09.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
68582025-12-16T03:03:09.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
68592025-12-16T03:03:09.818ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
68602025-12-16T03:03:09.818ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
68612025-12-16T03:03:09.818ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
68622025-12-16T03:03:09.819ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
68632025-12-16T03:03:09.819ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
68642025-12-16T03:03:09.819ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
68652025-12-16T03:03:09.819ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
68662025-12-16T03:03:09.819ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
68672025-12-16T03:03:09.819ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68682025-12-16T03:03:09.819ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
68692025-12-16T03:03:09.819ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
68702025-12-16T03:03:09.820ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
68712025-12-16T03:03:09.820ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68722025-12-16T03:03:09.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68732025-12-16T03:03:09.822ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
68742025-12-16T03:03:09.822ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
68752025-12-16T03:03:09.823ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
68762025-12-16T03:03:09.824ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
68772025-12-16T03:03:09.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
68782025-12-16T03:03:09.826ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68792025-12-16T03:03:09.829ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
68802025-12-16T03:03:09.829ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
68812025-12-16T03:03:09.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
68822025-12-16T03:03:09.831ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68832025-12-16T03:03:09.836ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
68842025-12-16T03:03:09.836ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
68852025-12-16T03:03:09.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
68862025-12-16T03:03:09.856ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68872025-12-16T03:03:09.863ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
68882025-12-16T03:03:09.863ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
68892025-12-16T03:03:09.863ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
68902025-12-16T03:03:09.863ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68912025-12-16T03:03:09.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68922025-12-16T03:03:09.864ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "64e7b340-44ad-49b7-9cbd-5a1bba4cd6b8", "date": "Tue, 16 Dec 2025 03:03:09 GMT"} })
68932025-12-16T03:03:09.864ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
68942025-12-16T03:03:09.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
68952025-12-16T03:03:09.867ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
68962025-12-16T03:03:09.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
68972025-12-16T03:03:09.871ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }], Name("external-dns"): [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
68982025-12-16T03:03:09.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68992025-12-16T03:03:09.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69002025-12-16T03:03:09.872ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
69012025-12-16T03:03:09.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
69022025-12-16T03:03:09.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69032025-12-16T03:03:09.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69042025-12-16T03:03:09.876ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
69052025-12-16T03:03:09.876ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
69062025-12-16T03:03:09.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69072025-12-16T03:03:09.877ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
69082025-12-16T03:03:09.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69092025-12-16T03:03:09.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69102025-12-16T03:03:09.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69112025-12-16T03:03:09.878ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "364914eb-1c7d-4742-9f97-ce750988ac35", "date": "Tue, 16 Dec 2025 03:03:09 GMT"} })
69122025-12-16T03:03:09.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69132025-12-16T03:03:09.879ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
69142025-12-16T03:03:09.879ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
69152025-12-16T03:03:09.879ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
69162025-12-16T03:03:09.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69172025-12-16T03:03:09.880ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 199.270714ms iteration = 14 status = {}
69182025-12-16T03:03:09.883ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
69192025-12-16T03:03:09.884ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "c115ac15-8467-49c9-b34d-83a32170a248", "content-length": "691", "date": "Tue, 16 Dec 2025 03:03:09 GMT"} })
69202025-12-16T03:03:09.884ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
69212025-12-16T03:03:09.884ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
69222025-12-16T03:03:09.884ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
69232025-12-16T03:03:09.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
69242025-12-16T03:03:09.885ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 673.771157ms iteration = 9 status = {}
69252025-12-16T03:03:09.941ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69262025-12-16T03:03:09.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69272025-12-16T03:03:09.945ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69282025-12-16T03:03:09.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69292025-12-16T03:03:09.947ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69302025-12-16T03:03:09.951ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
69312025-12-16T03:03:09.951ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69322025-12-16T03:03:09.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69332025-12-16T03:03:09.954ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69342025-12-16T03:03:09.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69352025-12-16T03:03:09.957ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69362025-12-16T03:03:09.960ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
69372025-12-16T03:03:09.960ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69382025-12-16T03:03:09.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69392025-12-16T03:03:09.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69402025-12-16T03:03:09.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69412025-12-16T03:03:09.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69422025-12-16T03:03:09.970ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), time_created: 2025-12-16T02:59:15.195809Z, time_modified: 2025-12-16T02:59:15.195809Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
69432025-12-16T03:03:09.970ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
69442025-12-16T03:03:09.970ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
69452025-12-16T03:03:09.970ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69462025-12-16T03:03:09.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69472025-12-16T03:03:09.973ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69482025-12-16T03:03:09.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69492025-12-16T03:03:09.975ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69502025-12-16T03:03:09.978ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
69512025-12-16T03:03:09.978ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69522025-12-16T03:03:09.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69532025-12-16T03:03:10.010ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69542025-12-16T03:03:10.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69552025-12-16T03:03:10.014ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69562025-12-16T03:03:10.014ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69572025-12-16T03:03:10.016ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69582025-12-16T03:03:10.020ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
69592025-12-16T03:03:10.020ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
69602025-12-16T03:03:10.021ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "b4404429-781f-425a-81b9-6e136e55c2c6", "date": "Tue, 16 Dec 2025 03:03:10 GMT"} })
69612025-12-16T03:03:10.026ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
69622025-12-16T03:03:10.026ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 97dbb60b-ce91-4d1d-9df6-232a11e8c982 step component = Dns step id = Ensure
69632025-12-16T03:03:10.026ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
69642025-12-16T03:03:10.026ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69652025-12-16T03:03:10.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69662025-12-16T03:03:10.027ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69672025-12-16T03:03:10.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69682025-12-16T03:03:10.029ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69692025-12-16T03:03:10.035ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
69702025-12-16T03:03:10.035ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
69712025-12-16T03:03:10.035ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 945e4164-27c9-41f8-971d-0d45c67d8fe7 parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
69722025-12-16T03:03:10.035ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69732025-12-16T03:03:10.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69742025-12-16T03:03:10.036ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
69752025-12-16T03:03:10.038ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69762025-12-16T03:03:10.041ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
69772025-12-16T03:03:10.041ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
69782025-12-16T03:03:10.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
69792025-12-16T03:03:10.045ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
69802025-12-16T03:03:10.045ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
69812025-12-16T03:03:10.045ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 835.232172ms iteration = 13 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
69822025-12-16T03:03:10.099ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
69832025-12-16T03:03:10.099ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
69842025-12-16T03:03:10.099ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
69852025-12-16T03:03:10.099ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69862025-12-16T03:03:10.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69872025-12-16T03:03:10.101ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69882025-12-16T03:03:10.106ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
69892025-12-16T03:03:10.106ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69902025-12-16T03:03:10.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69912025-12-16T03:03:10.111ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
69922025-12-16T03:03:10.111ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
69932025-12-16T03:03:10.111ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69942025-12-16T03:03:10.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69952025-12-16T03:03:10.113ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69962025-12-16T03:03:10.118ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
69972025-12-16T03:03:10.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69982025-12-16T03:03:10.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69992025-12-16T03:03:10.122ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
70002025-12-16T03:03:10.122ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
70012025-12-16T03:03:10.122ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
70022025-12-16T03:03:10.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70032025-12-16T03:03:10.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70042025-12-16T03:03:10.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70052025-12-16T03:03:10.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70062025-12-16T03:03:10.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70072025-12-16T03:03:10.131ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
70082025-12-16T03:03:10.132ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
70092025-12-16T03:03:10.132ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
70102025-12-16T03:03:10.132ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
70112025-12-16T03:03:10.132ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70122025-12-16T03:03:10.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70132025-12-16T03:03:10.134ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70142025-12-16T03:03:10.138ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
70152025-12-16T03:03:10.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70162025-12-16T03:03:10.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70172025-12-16T03:03:10.141ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
70182025-12-16T03:03:10.141ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
70192025-12-16T03:03:10.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70202025-12-16T03:03:10.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70212025-12-16T03:03:10.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70222025-12-16T03:03:10.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
70232025-12-16T03:03:10.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70242025-12-16T03:03:10.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70252025-12-16T03:03:10.148ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
70262025-12-16T03:03:10.148ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
70272025-12-16T03:03:10.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70282025-12-16T03:03:10.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70292025-12-16T03:03:10.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70302025-12-16T03:03:10.152ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
70312025-12-16T03:03:10.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70322025-12-16T03:03:10.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70332025-12-16T03:03:10.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70342025-12-16T03:03:10.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70352025-12-16T03:03:10.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70362025-12-16T03:03:10.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70372025-12-16T03:03:10.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70382025-12-16T03:03:10.160ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
70392025-12-16T03:03:10.161ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
70402025-12-16T03:03:10.161ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
70412025-12-16T03:03:10.161ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 97dbb60b-ce91-4d1d-9df6-232a11e8c982 step component = OmicronZones step id = Cleanup
70422025-12-16T03:03:10.161ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 97dbb60b-ce91-4d1d-9df6-232a11e8c982 step component = OmicronZones step id = Cleanup
70432025-12-16T03:03:10.161ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 97dbb60b-ce91-4d1d-9df6-232a11e8c982 step component = PhysicalDisks step id = Cleanup
70442025-12-16T03:03:10.161ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 97dbb60b-ce91-4d1d-9df6-232a11e8c982 step component = Clickhouse step id = Ensure
70452025-12-16T03:03:10.161ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 97dbb60b-ce91-4d1d-9df6-232a11e8c982 step component = Clickhouse step id = Ensure
70462025-12-16T03:03:10.192ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
70472025-12-16T03:03:10.197ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "8cdd8f2d-4a07-4054-8c0b-8300dc22c2bf", "date": "Tue, 16 Dec 2025 03:03:10 GMT"} })
70482025-12-16T03:03:10.197ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
70492025-12-16T03:03:10.197ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
70502025-12-16T03:03:10.197ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "86a31f04-3295-479e-9130-77415b6c056a", "content-length": "682", "date": "Tue, 16 Dec 2025 03:03:10 GMT"} })
70512025-12-16T03:03:10.197ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
70522025-12-16T03:03:10.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
70532025-12-16T03:03:10.198ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "3325e114-1f8a-405f-95ca-32103baf28d5", "date": "Tue, 16 Dec 2025 03:03:10 GMT"} })
70542025-12-16T03:03:10.200ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 97dbb60b-ce91-4d1d-9df6-232a11e8c982 step component = SupportBundles step id = Cleanup
70552025-12-16T03:03:10.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70562025-12-16T03:03:10.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70572025-12-16T03:03:10.201ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
70582025-12-16T03:03:10.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
70592025-12-16T03:03:10.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70602025-12-16T03:03:10.204ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
70612025-12-16T03:03:10.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70622025-12-16T03:03:10.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70632025-12-16T03:03:10.204ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
70642025-12-16T03:03:10.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
70652025-12-16T03:03:10.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70662025-12-16T03:03:10.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70672025-12-16T03:03:10.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70682025-12-16T03:03:10.207ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
70692025-12-16T03:03:10.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
70702025-12-16T03:03:10.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
70712025-12-16T03:03:10.214ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.003416793s iteration = 9 status = {}
70722025-12-16T03:03:10.256ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 97dbb60b-ce91-4d1d-9df6-232a11e8c982 step component = OmicronZones step id = Cleanup
70732025-12-16T03:03:10.256ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
70742025-12-16T03:03:10.256ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
70752025-12-16T03:03:10.256ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70762025-12-16T03:03:10.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70772025-12-16T03:03:10.258ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70782025-12-16T03:03:10.260ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
70792025-12-16T03:03:10.260ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70802025-12-16T03:03:10.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70812025-12-16T03:03:10.295ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
70822025-12-16T03:03:10.295ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(a91734db-9ac7-4793-9c32-ddbb21f8d986 (blueprint), false)
70832025-12-16T03:03:10.295ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 97dbb60b-ce91-4d1d-9df6-232a11e8c982 step component = Cockroach step id = Ensure
70842025-12-16T03:03:10.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70852025-12-16T03:03:10.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70862025-12-16T03:03:10.315ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
70872025-12-16T03:03:10.316ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 97dbb60b-ce91-4d1d-9df6-232a11e8c982 step component = MgsUpdates step id = Ensure
70882025-12-16T03:03:10.316ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 14 reason = Dependency
70892025-12-16T03:03:10.316ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 6 reason = Dependency
70902025-12-16T03:03:10.316ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
70912025-12-16T03:03:10.316ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
70922025-12-16T03:03:10.316ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.145648546s iteration = 6 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"7c5c4d37-4c6f-4dce-8a58-835cb6c53d87","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"7c5c4d37-4c6f-4dce-8a58-835cb6c53d87","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":4520,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":110468540,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":110468500,"secs":0}},"event_index":1,"execution_id":"7c5c4d37-4c6f-4dce-8a58-835cb6c53d87","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":110492930,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":97763439,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":97763399,"secs":0}},"event_index":2,"execution_id":"7c5c4d37-4c6f-4dce-8a58-835cb6c53d87","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":208278479,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":132259865,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":132259825,"secs":0}},"event_index":3,"execution_id":"7c5c4d37-4c6f-4dce-8a58-835cb6c53d87","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":340555364,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":134267132,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":134267072,"secs":0}},"event_index":4,"execution_id":"7c5c4d37-4c6f-4dce-8a58-835cb6c53d87","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":474847766,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":380786845,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":380786795,"secs":0}},"event_index":5,"execution_id":"7c5c4d37-4c6f-4dce-8a58-835cb6c53d87","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":855653541,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":135400626,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":135400586,"secs":0}},"event_index":6,"execution_id":"7c5c4d37-4c6f-4dce-8a58-835cb6c53d87","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":991072247,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3960,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":3920,"secs":0}},"event_index":7,"execution_id":"7c5c4d37-4c6f-4dce-8a58-835cb6c53d87","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":991090447,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2780,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2750,"secs":0}},"event_index":8,"execution_id":"7c5c4d37-4c6f-4dce-8a58-835cb6c53d87","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":991096107,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3570,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3530,"secs":0}},"event_index":9,"execution_id":"7c5c4d37-4c6f-4dce-8a58-835cb6c53d87","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":991101397,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2230,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":2190,"secs":0}},"event_index":10,"execution_id":"7c5c4d37-4c6f-4dce-8a58-835cb6c53d87","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":991105887,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":38532039,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":38532008,"secs":0}},"event_index":11,"execution_id":"7c5c4d37-4c6f-4dce-8a58-835cb6c53d87","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":29640256,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":55943420,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":55943380,"secs":0}},"event_index":12,"execution_id":"7c5c4d37-4c6f-4dce-8a58-835cb6c53d87","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":85597294,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":39135176,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":39135136,"secs":0}},"event_index":13,"execution_id":"7c5c4d37-4c6f-4dce-8a58-835cb6c53d87","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":124755221,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":20514580,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":20514530,"secs":0}},"event_index":14,"execution_id":"7c5c4d37-4c6f-4dce-8a58-835cb6c53d87","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":145282201,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5600,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5550,"secs":0}},"event_index":15,"execution_id":"7c5c4d37-4c6f-4dce-8a58-835cb6c53d87","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":145295881,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986"}
70932025-12-16T03:03:10.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
70942025-12-16T03:03:10.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
70952025-12-16T03:03:10.318ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70962025-12-16T03:03:10.318ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70972025-12-16T03:03:10.321ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
70982025-12-16T03:03:10.321ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
70992025-12-16T03:03:10.321ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
71002025-12-16T03:03:10.321ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
71012025-12-16T03:03:10.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
71022025-12-16T03:03:10.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
71032025-12-16T03:03:10.323ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
71042025-12-16T03:03:10.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
71052025-12-16T03:03:10.325ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71062025-12-16T03:03:10.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
71072025-12-16T03:03:10.327ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
71082025-12-16T03:03:10.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
71092025-12-16T03:03:10.372ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
71102025-12-16T03:03:10.372ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 56.194352ms iteration = 14 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
71112025-12-16T03:03:10.589ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(cd9f1d61-49c8-4ef5-9e52-7e084cb4a60e, 0), (fecebf81-1ff4-4296-8099-25ffd5550e29, 0), (3961f0bb-749a-4198-8468-d62cb19df0b8, 0), (2cd1056d-9147-4b28-acdd-ae58d67a929e, 0)] collection_id = cd9f1d61-49c8-4ef5-9e52-7e084cb4a60e
71122025-12-16T03:03:10.589ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
71132025-12-16T03:03:10.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
71142025-12-16T03:03:10.859ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = cd9f1d61-49c8-4ef5-9e52-7e084cb4a60e file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
71152025-12-16T03:03:10.859ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
71162025-12-16T03:03:10.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
71172025-12-16T03:03:11.024ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(3961f0bb-749a-4198-8468-d62cb19df0b8, 0), (2cd1056d-9147-4b28-acdd-ae58d67a929e, 0), (cd59d5cd-cde1-4196-b850-0ce71d63843c, 0)]
71182025-12-16T03:03:11.025ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1198, tv_nsec: 640091670 } })
71192025-12-16T03:03:11.056ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("20488683-5778-4090-b36e-e6f0d80a682b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("35dbaad1-8536-44a0-8a0a-35d7c39dcfbb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b57fa226-2428-4a54-b85d-60409e28b484.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("d812a3e2-48ae-4812-a1cc-7eb7623e4e00.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::5)) }], valid_until: Instant { tv_sec: 1198, tv_nsec: 671982444 } })
71202025-12-16T03:03:11.057ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
71212025-12-16T03:03:11.057ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
71222025-12-16T03:03:11.057ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
71232025-12-16T03:03:11.058ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "629ffe8b-93cc-4d2d-baf0-8ea5a6a57e4c", "content-length": "476", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71242025-12-16T03:03:11.058ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
71252025-12-16T03:03:11.058ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2076ff51-30e0-40ed-828d-2cad9a35f8f3", "content-length": "734", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71262025-12-16T03:03:11.058ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
71272025-12-16T03:03:11.059ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ec9dbf2-24e0-438e-b5a6-ca4bef164f1a", "content-length": "111", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71282025-12-16T03:03:11.059ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
71292025-12-16T03:03:11.059ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e62c2b7-1d4b-4ecb-8b6d-f754607c66dc", "content-length": "111", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71302025-12-16T03:03:11.059ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
71312025-12-16T03:03:11.059ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c6e15f4-c14a-4a0d-bb24-a5ca6e228871", "content-length": "163", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71322025-12-16T03:03:11.059ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
71332025-12-16T03:03:11.060ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c51a4ba2-64fe-4a16-90f1-afb153eaca21", "content-length": "163", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71342025-12-16T03:03:11.060ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
71352025-12-16T03:03:11.060ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2aff9b2e-62d4-4096-8342-598aa4b574e7", "content-length": "166", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71362025-12-16T03:03:11.060ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
71372025-12-16T03:03:11.061ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e95a2780-d6c1-44ff-a1b9-2c70aebce1c3", "content-length": "166", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71382025-12-16T03:03:11.061ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
71392025-12-16T03:03:11.061ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfdcd00c-57fd-4ca7-9d7a-9fb270ca3e74", "content-length": "702", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71402025-12-16T03:03:11.061ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
71412025-12-16T03:03:11.061ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "62abe33b-2203-4d4b-b341-94a275d6d7d3", "content-length": "727", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71422025-12-16T03:03:11.061ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
71432025-12-16T03:03:11.061ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8c0673c-cdf8-4260-8d02-c22eb23ddc35", "content-length": "729", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71442025-12-16T03:03:11.061ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
71452025-12-16T03:03:11.061ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8258984f-2832-446a-971f-7084a359a7b1", "content-length": "728", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71462025-12-16T03:03:11.061ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
71472025-12-16T03:03:11.062ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c68befb8-1efe-4136-946f-438420adf049", "content-length": "722", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71482025-12-16T03:03:11.062ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
71492025-12-16T03:03:11.062ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 281 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = fde39b88-00b0-4f81-9bd3-87442a1f5071 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
71502025-12-16T03:03:11.062ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "319341d5-b841-4244-956c-b4904aaf5907", "content-length": "10", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71512025-12-16T03:03:11.062ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
71522025-12-16T03:03:11.062ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4473641b-f3f0-4f03-b13a-730a595024a3", "content-length": "144", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71532025-12-16T03:03:11.062ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
71542025-12-16T03:03:11.063ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5251e413-094a-4c5e-948b-2d50cc0d1b7d", "content-length": "147", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71552025-12-16T03:03:11.063ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
71562025-12-16T03:03:11.063ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc7ac0e7-f43c-408e-bbf9-028d514827ca", "content-length": "109", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71572025-12-16T03:03:11.063ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
71582025-12-16T03:03:11.063ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e6c5a63-408f-4c01-9e20-3a646008cb5f", "content-length": "109", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71592025-12-16T03:03:11.063ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
71602025-12-16T03:03:11.064ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "67b0f7ae-c640-4462-9836-2b0118f2dcc2", "content-length": "163", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71612025-12-16T03:03:11.064ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
71622025-12-16T03:03:11.064ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c82a31c9-e2b9-444b-a085-bf0930d94838", "content-length": "163", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71632025-12-16T03:03:11.064ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
71642025-12-16T03:03:11.065ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4dcce97c-f7bb-4340-af8b-fa05f25b68ca", "content-length": "166", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71652025-12-16T03:03:11.065ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
71662025-12-16T03:03:11.065ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e0dfda2-2d8b-4ea6-a83d-b6aada8eb17e", "content-length": "166", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71672025-12-16T03:03:11.065ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
71682025-12-16T03:03:11.065ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "65bb4bc6-ca98-43f7-a240-1b1ed83f949d", "content-length": "702", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71692025-12-16T03:03:11.065ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
71702025-12-16T03:03:11.066ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d07beae8-4e98-48bf-bb43-59044b17c3cc", "content-length": "727", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71712025-12-16T03:03:11.066ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
71722025-12-16T03:03:11.066ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b32fd722-0934-433a-bc93-0672cb98531c", "content-length": "729", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71732025-12-16T03:03:11.066ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
71742025-12-16T03:03:11.066ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c53dea0-7895-4652-bc8c-f72b80a9a506", "content-length": "728", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71752025-12-16T03:03:11.066ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
71762025-12-16T03:03:11.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
71772025-12-16T03:03:11.068ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71782025-12-16T03:03:11.070ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
71792025-12-16T03:03:11.070ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
71802025-12-16T03:03:11.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
71812025-12-16T03:03:11.072ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71822025-12-16T03:03:11.075ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
71832025-12-16T03:03:11.075ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
71842025-12-16T03:03:11.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
71852025-12-16T03:03:11.111ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
71862025-12-16T03:03:11.111ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
71872025-12-16T03:03:11.191ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "3eb9bd75-92c4-4542-9ee9-5243ae35184c", "content-length": "95612", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71882025-12-16T03:03:11.193ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
71892025-12-16T03:03:11.193ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
71902025-12-16T03:03:11.193ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
71912025-12-16T03:03:11.193ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
71922025-12-16T03:03:11.193ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
71932025-12-16T03:03:11.193ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
71942025-12-16T03:03:11.193ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
71952025-12-16T03:03:11.193ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
71962025-12-16T03:03:11.193ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b141316c-a39d-45cf-b58f-9f4186afc7d3", "content-length": "64", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71972025-12-16T03:03:11.193ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
71982025-12-16T03:03:11.193ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5c99334-bc89-439b-9598-4c73850f27a9", "content-length": "64", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
71992025-12-16T03:03:11.193ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3d6334c-5bd3-4716-ac1d-39b62da45a73", "content-length": "64", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
72002025-12-16T03:03:11.193ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "49031f32-2ae0-480e-8bb5-d53acbe0571a", "content-length": "64", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
72012025-12-16T03:03:11.193ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
72022025-12-16T03:03:11.193ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
72032025-12-16T03:03:11.193ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
72042025-12-16T03:03:11.193ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "11bdb1a7-a248-4bc2-b267-0be623588908", "content-length": "64", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
72052025-12-16T03:03:11.193ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
72062025-12-16T03:03:11.230ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9d032ee-c1b8-4d57-954d-079b3ff5d841", "content-length": "424690", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
72072025-12-16T03:03:11.234ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "446a9c8a-61da-4954-8eec-112af662c3e6", "content-length": "392441", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
72082025-12-16T03:03:11.235ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "511add49-9f35-41b5-ab47-da9205de16c9", "content-length": "395820", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
72092025-12-16T03:03:11.243ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "624d3f87-f6fe-4b81-a95d-829f84dc59ed", "content-length": "392011", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
72102025-12-16T03:03:11.263ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "68f05312-8969-409c-8e74-05aa15475b55", "content-length": "395686", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
72112025-12-16T03:03:11.265ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
72122025-12-16T03:03:11.265ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
72132025-12-16T03:03:11.265ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
72142025-12-16T03:03:11.265ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
72152025-12-16T03:03:11.265ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
72162025-12-16T03:03:11.296ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c (omicron_zone)
72172025-12-16T03:03:11.296ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
72182025-12-16T03:03:11.299ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "06bc2ae4-7ecc-48ff-87af-ffe6b55ca629", "content-length": "124", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
72192025-12-16T03:03:11.302ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
72202025-12-16T03:03:11.400ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
72212025-12-16T03:03:11.400ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
72222025-12-16T03:03:11.401ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "61c80dc6-86e0-406e-a6f3-8cb465d5dbb3", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
72232025-12-16T03:03:11.401ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
72242025-12-16T03:03:11.402ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
72252025-12-16T03:03:11.402ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
72262025-12-16T03:03:11.403ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c2fa029-67f0-4fdd-a0cf-d3567ccc8a23", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
72272025-12-16T03:03:11.403ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
72282025-12-16T03:03:11.404ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
72292025-12-16T03:03:11.404ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
72302025-12-16T03:03:11.405ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "2407b139-0d2c-4f2b-961a-0245550df643", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:03:11 GMT"} })
72312025-12-16T03:03:11.405ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
72322025-12-16T03:03:11.406ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
72332025-12-16T03:03:11.406ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
72342025-12-16T03:03:11.408ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
72352025-12-16T03:03:11.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
72362025-12-16T03:03:11.410ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72372025-12-16T03:03:11.413ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
72382025-12-16T03:03:11.413ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
72392025-12-16T03:03:11.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
72402025-12-16T03:03:11.655ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 58e21b5f-57d6-4b9f-861f-24cabd63d443 file = nexus/db-queries/src/db/datastore/inventory.rs:1613
72412025-12-16T03:03:11.655ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 58e21b5f-57d6-4b9f-861f-24cabd63d443 time_started = 2025-12-16 03:03:11.057623 UTC
72422025-12-16T03:03:11.655ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 17 reason = Dependency
72432025-12-16T03:03:11.655ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
72442025-12-16T03:03:11.655ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 23 reason = Dependency
72452025-12-16T03:03:11.655ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 15.44\u{b5}s iteration = 17 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
72462025-12-16T03:03:11.655ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
72472025-12-16T03:03:11.655ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.33936416s iteration = 6 status = {"collection_id":"58e21b5f-57d6-4b9f-861f-24cabd63d443","time_done":"2025-12-16 03:03:11.406435 UTC","time_started":"2025-12-16 03:03:11.057623 UTC"}
72482025-12-16T03:03:11.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
72492025-12-16T03:03:11.657ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72502025-12-16T03:03:11.661ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
72512025-12-16T03:03:11.661ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
72522025-12-16T03:03:11.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
72532025-12-16T03:03:11.676ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
72542025-12-16T03:03:11.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
72552025-12-16T03:03:11.740ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 14 reason = Dependency
72562025-12-16T03:03:11.740ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 84.573271ms iteration = 23 status = {"Loaded":{"collection_id":"b48a1e8c-4c75-4571-b0b8-834af8e60fa2","time_loaded":"2025-12-16T03:03:11.655455225Z","time_started":"2025-12-16T03:03:11.058329Z"}}
72572025-12-16T03:03:11.740ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 16 reason = Dependency
72582025-12-16T03:03:11.740ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 18 reason = Dependency
72592025-12-16T03:03:11.740ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
72602025-12-16T03:03:11.740ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 7.26\u{b5}s iteration = 18 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
72612025-12-16T03:03:11.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72622025-12-16T03:03:11.740ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 8 reason = Dependency
72632025-12-16T03:03:11.740ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
72642025-12-16T03:03:11.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
72652025-12-16T03:03:11.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72662025-12-16T03:03:11.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
72672025-12-16T03:03:11.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
72682025-12-16T03:03:11.741ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72692025-12-16T03:03:11.742ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72702025-12-16T03:03:11.742ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72712025-12-16T03:03:11.745ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
72722025-12-16T03:03:11.745ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
72732025-12-16T03:03:11.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
72742025-12-16T03:03:11.745ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
72752025-12-16T03:03:11.745ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72762025-12-16T03:03:11.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72772025-12-16T03:03:11.745ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
72782025-12-16T03:03:11.745ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
72792025-12-16T03:03:11.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
72802025-12-16T03:03:11.747ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72812025-12-16T03:03:11.747ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72822025-12-16T03:03:11.747ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
72832025-12-16T03:03:11.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
72842025-12-16T03:03:11.749ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72852025-12-16T03:03:11.750ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
72862025-12-16T03:03:11.750ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72872025-12-16T03:03:11.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72882025-12-16T03:03:11.751ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
72892025-12-16T03:03:11.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
72902025-12-16T03:03:11.752ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
72912025-12-16T03:03:11.752ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
72922025-12-16T03:03:11.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
72932025-12-16T03:03:11.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
72942025-12-16T03:03:11.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
72952025-12-16T03:03:11.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
72962025-12-16T03:03:11.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72972025-12-16T03:03:11.758ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
72982025-12-16T03:03:11.758ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
72992025-12-16T03:03:11.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
73002025-12-16T03:03:11.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73012025-12-16T03:03:11.760ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 20.396558ms iteration = 16 status = {"physical_disks_added":0}
73022025-12-16T03:03:11.761ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
73032025-12-16T03:03:11.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
73042025-12-16T03:03:11.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
73052025-12-16T03:03:11.764ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
73062025-12-16T03:03:11.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
73072025-12-16T03:03:11.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
73082025-12-16T03:03:11.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73092025-12-16T03:03:11.768ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
73102025-12-16T03:03:11.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
73112025-12-16T03:03:11.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
73122025-12-16T03:03:11.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73132025-12-16T03:03:11.771ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
73142025-12-16T03:03:11.771ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
73152025-12-16T03:03:11.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
73162025-12-16T03:03:11.773ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
73172025-12-16T03:03:11.773ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 33.705717ms iteration = 8 status = {"blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","inventory_collection_id":"b48a1e8c-4c75-4571-b0b8-834af8e60fa2","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
73182025-12-16T03:03:11.829ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73192025-12-16T03:03:11.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73202025-12-16T03:03:11.831ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73212025-12-16T03:03:11.833ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
73222025-12-16T03:03:11.833ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73232025-12-16T03:03:11.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73242025-12-16T03:03:11.835ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73252025-12-16T03:03:11.837ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
73262025-12-16T03:03:11.837ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73272025-12-16T03:03:11.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73282025-12-16T03:03:11.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73292025-12-16T03:03:11.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73302025-12-16T03:03:11.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73312025-12-16T03:03:11.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73322025-12-16T03:03:11.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73332025-12-16T03:03:11.888ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
73342025-12-16T03:03:11.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73352025-12-16T03:03:11.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73362025-12-16T03:03:11.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73372025-12-16T03:03:11.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73382025-12-16T03:03:11.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73392025-12-16T03:03:11.895ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
73402025-12-16T03:03:11.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73412025-12-16T03:03:11.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73422025-12-16T03:03:11.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73432025-12-16T03:03:11.899ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
73442025-12-16T03:03:11.899ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73452025-12-16T03:03:11.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73462025-12-16T03:03:11.901ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73472025-12-16T03:03:11.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73482025-12-16T03:03:11.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73492025-12-16T03:03:11.905ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
73502025-12-16T03:03:11.905ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73512025-12-16T03:03:11.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73522025-12-16T03:03:11.907ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73532025-12-16T03:03:11.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73542025-12-16T03:03:11.910ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73552025-12-16T03:03:11.914ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
73562025-12-16T03:03:11.914ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73572025-12-16T03:03:11.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73582025-12-16T03:03:11.916ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73592025-12-16T03:03:11.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73602025-12-16T03:03:11.918ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73612025-12-16T03:03:11.921ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
73622025-12-16T03:03:11.921ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73632025-12-16T03:03:11.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73642025-12-16T03:03:11.939ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73652025-12-16T03:03:11.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73662025-12-16T03:03:11.941ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73672025-12-16T03:03:11.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73682025-12-16T03:03:11.943ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73692025-12-16T03:03:11.946ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
73702025-12-16T03:03:11.946ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73712025-12-16T03:03:11.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73722025-12-16T03:03:11.948ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73732025-12-16T03:03:11.951ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
73742025-12-16T03:03:11.951ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73752025-12-16T03:03:11.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73762025-12-16T03:03:11.954ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73772025-12-16T03:03:11.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73782025-12-16T03:03:11.956ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73792025-12-16T03:03:11.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73802025-12-16T03:03:11.958ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73812025-12-16T03:03:11.961ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
73822025-12-16T03:03:11.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73832025-12-16T03:03:11.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73842025-12-16T03:03:11.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73852025-12-16T03:03:11.966ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
73862025-12-16T03:03:11.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73872025-12-16T03:03:11.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73882025-12-16T03:03:11.969ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73892025-12-16T03:03:11.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73902025-12-16T03:03:11.970ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73912025-12-16T03:03:11.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73922025-12-16T03:03:11.971ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73932025-12-16T03:03:11.974ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
73942025-12-16T03:03:11.975ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73952025-12-16T03:03:11.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73962025-12-16T03:03:11.976ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73972025-12-16T03:03:11.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73982025-12-16T03:03:11.977ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73992025-12-16T03:03:11.981ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
74002025-12-16T03:03:11.981ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
74012025-12-16T03:03:11.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
74022025-12-16T03:03:11.984ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
74032025-12-16T03:03:11.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
74042025-12-16T03:03:11.985ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74052025-12-16T03:03:11.988ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
74062025-12-16T03:03:11.988ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
74072025-12-16T03:03:11.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
74082025-12-16T03:03:11.990ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
74092025-12-16T03:03:11.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
74102025-12-16T03:03:11.991ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74112025-12-16T03:03:11.994ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
74122025-12-16T03:03:11.994ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
74132025-12-16T03:03:11.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
74142025-12-16T03:03:11.996ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
74152025-12-16T03:03:11.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
74162025-12-16T03:03:11.998ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74172025-12-16T03:03:12.000ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
74182025-12-16T03:03:12.000ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
74192025-12-16T03:03:12.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
74202025-12-16T03:03:12.002ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
74212025-12-16T03:03:12.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
74222025-12-16T03:03:12.003ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74232025-12-16T03:03:12.005ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
74242025-12-16T03:03:12.005ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
74252025-12-16T03:03:12.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
74262025-12-16T03:03:12.044ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
74272025-12-16T03:03:12.044ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
74282025-12-16T03:03:12.044ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = e56dac1f-62cc-49c2-9b64-3e38db2d63c3 parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
74292025-12-16T03:03:12.044ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
74302025-12-16T03:03:12.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
74312025-12-16T03:03:12.046ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74322025-12-16T03:03:12.049ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
74332025-12-16T03:03:12.049ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
74342025-12-16T03:03:12.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
74352025-12-16T03:03:12.051ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
74362025-12-16T03:03:12.051ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
74372025-12-16T03:03:12.052ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 312.024133ms iteration = 14 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
74382025-12-16T03:03:14.211ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 50 reason = Timeout
74392025-12-16T03:03:14.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
74402025-12-16T03:03:14.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
74412025-12-16T03:03:14.213ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74422025-12-16T03:03:14.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
74432025-12-16T03:03:14.216ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
74442025-12-16T03:03:14.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
74452025-12-16T03:03:14.218ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
74462025-12-16T03:03:14.218ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.470875ms iteration = 50 status = {"config_updated":false}
74472025-12-16T03:03:19.211ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 26 reason = Timeout
74482025-12-16T03:03:19.211ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 51 reason = Timeout
74492025-12-16T03:03:19.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74502025-12-16T03:03:19.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
74512025-12-16T03:03:19.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
74522025-12-16T03:03:19.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
74532025-12-16T03:03:19.211ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 26 reason = Timeout
74542025-12-16T03:03:19.211ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
74552025-12-16T03:03:19.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
74562025-12-16T03:03:19.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74572025-12-16T03:03:19.213ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74582025-12-16T03:03:19.213ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
74592025-12-16T03:03:19.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
74602025-12-16T03:03:19.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
74612025-12-16T03:03:19.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74622025-12-16T03:03:19.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
74632025-12-16T03:03:19.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
74642025-12-16T03:03:19.217ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
74652025-12-16T03:03:19.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
74662025-12-16T03:03:19.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74672025-12-16T03:03:19.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
74682025-12-16T03:03:19.219ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
74692025-12-16T03:03:19.219ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.068832ms iteration = 51 status = {"config_updated":false}
74702025-12-16T03:03:19.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74712025-12-16T03:03:19.223ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
74722025-12-16T03:03:19.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74732025-12-16T03:03:19.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
74742025-12-16T03:03:19.240ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
74752025-12-16T03:03:19.240ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.091089ms iteration = 26 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
74762025-12-16T03:03:19.257ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 45.585818ms iteration = 26 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
74772025-12-16T03:03:21.055ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = f7d17d43-6f5b-469f-bd09-d999f79e1b03 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending version = 1.0.0
74782025-12-16T03:03:21.055ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = f7d17d43-6f5b-469f-bd09-d999f79e1b03 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
74792025-12-16T03:03:21.056ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = f7d17d43-6f5b-469f-bd09-d999f79e1b03 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
74802025-12-16T03:03:21.059ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 3151 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = f7d17d43-6f5b-469f-bd09-d999f79e1b03 response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending
74812025-12-16T03:03:21.059ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 002554cb-31be-4af4-8072-18c412ba6aaa uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D version = 1.0.0
74822025-12-16T03:03:21.059ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 002554cb-31be-4af4-8072-18c412ba6aaa roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
74832025-12-16T03:03:21.059ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 002554cb-31be-4af4-8072-18c412ba6aaa resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
74842025-12-16T03:03:21.062ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 2784 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 002554cb-31be-4af4-8072-18c412ba6aaa response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D
74852025-12-16T03:03:21.062ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 284 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = fa17efe2-4d62-45a6-b067-9468f1242960 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
74862025-12-16T03:03:24.211ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 24 reason = Timeout
74872025-12-16T03:03:24.211ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 52 reason = Timeout
74882025-12-16T03:03:24.211ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
74892025-12-16T03:03:24.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
74902025-12-16T03:03:24.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
74912025-12-16T03:03:24.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
74922025-12-16T03:03:24.213ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74932025-12-16T03:03:24.213ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74942025-12-16T03:03:24.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
74952025-12-16T03:03:24.217ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
74962025-12-16T03:03:24.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
74972025-12-16T03:03:24.217ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
74982025-12-16T03:03:24.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
74992025-12-16T03:03:24.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
75002025-12-16T03:03:24.219ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
75012025-12-16T03:03:24.219ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.88538ms iteration = 52 status = {"config_updated":false}
75022025-12-16T03:03:24.220ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = b48a1e8c-4c75-4571-b0b8-834af8e60fa2 original_time_started = 2025-12-16 03:03:11.058329 UTC
75032025-12-16T03:03:24.220ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.834062ms iteration = 24 status = {"Loaded":{"collection_id":"b48a1e8c-4c75-4571-b0b8-834af8e60fa2","time_loaded":"2025-12-16T03:03:24.211579845Z","time_started":"2025-12-16T03:03:11.058329Z"}}
75042025-12-16T03:03:24.229ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 18 reason = Timeout
75052025-12-16T03:03:24.229ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 18 reason = Timeout
75062025-12-16T03:03:24.229ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
75072025-12-16T03:03:24.229ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
75082025-12-16T03:03:24.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
75092025-12-16T03:03:24.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
75102025-12-16T03:03:24.230ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
75112025-12-16T03:03:24.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
75122025-12-16T03:03:24.231ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75132025-12-16T03:03:24.231ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75142025-12-16T03:03:24.233ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
75152025-12-16T03:03:24.234ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
75162025-12-16T03:03:24.234ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
75172025-12-16T03:03:24.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
75182025-12-16T03:03:24.236ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.638794ms iteration = 18 status = "NoSitrep"
75192025-12-16T03:03:24.236ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
75202025-12-16T03:03:24.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
75212025-12-16T03:03:24.237ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
75222025-12-16T03:03:24.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
75232025-12-16T03:03:24.239ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75242025-12-16T03:03:24.240ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
75252025-12-16T03:03:24.241ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
75262025-12-16T03:03:24.241ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 12.007012ms iteration = 18 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
75272025-12-16T03:03:29.211ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 27 reason = Timeout
75282025-12-16T03:03:29.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75292025-12-16T03:03:29.212ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 53 reason = Timeout
75302025-12-16T03:03:29.212ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
75312025-12-16T03:03:29.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
75322025-12-16T03:03:29.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
75332025-12-16T03:03:29.212ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 27 reason = Timeout
75342025-12-16T03:03:29.212ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
75352025-12-16T03:03:29.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
75362025-12-16T03:03:29.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75372025-12-16T03:03:29.214ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
75382025-12-16T03:03:29.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
75392025-12-16T03:03:29.214ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75402025-12-16T03:03:29.216ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.368362ms iteration = 27 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
75412025-12-16T03:03:29.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
75422025-12-16T03:03:29.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75432025-12-16T03:03:29.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
75442025-12-16T03:03:29.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
75452025-12-16T03:03:29.218ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
75462025-12-16T03:03:29.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
75472025-12-16T03:03:29.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75482025-12-16T03:03:29.220ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
75492025-12-16T03:03:29.220ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.438578ms iteration = 53 status = {"config_updated":false}
75502025-12-16T03:03:29.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
75512025-12-16T03:03:29.221ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75522025-12-16T03:03:29.224ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
75532025-12-16T03:03:29.224ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75542025-12-16T03:03:29.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
75552025-12-16T03:03:29.237ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
75562025-12-16T03:03:29.237ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.23261ms iteration = 27 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
75572025-12-16T03:03:31.063ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 359 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 917d6616-7786-4c5e-8c8e-6a47f5aa094e response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
75582025-12-16T03:03:34.211ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 54 reason = Timeout
75592025-12-16T03:03:34.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
75602025-12-16T03:03:34.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
75612025-12-16T03:03:34.213ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75622025-12-16T03:03:34.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
75632025-12-16T03:03:34.216ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
75642025-12-16T03:03:34.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
75652025-12-16T03:03:34.218ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
75662025-12-16T03:03:34.218ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.904657ms iteration = 54 status = {"config_updated":false}
75672025-12-16T03:03:36.055ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 3316bf25-1191-454a-ba52-6d6e3b3aedba uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending version = 1.0.0
75682025-12-16T03:03:36.056ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 3316bf25-1191-454a-ba52-6d6e3b3aedba roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
75692025-12-16T03:03:36.056ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 3316bf25-1191-454a-ba52-6d6e3b3aedba resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
75702025-12-16T03:03:36.059ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 3767 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 3316bf25-1191-454a-ba52-6d6e3b3aedba response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending
75712025-12-16T03:03:36.059ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 883fb40b-8729-4d41-a4a1-3593208453ac uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D version = 1.0.0
75722025-12-16T03:03:36.060ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 883fb40b-8729-4d41-a4a1-3593208453ac roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
75732025-12-16T03:03:36.060ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 883fb40b-8729-4d41-a4a1-3593208453ac resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
75742025-12-16T03:03:36.062ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 2564 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:41727 req_id = 883fb40b-8729-4d41-a4a1-3593208453ac response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D
75752025-12-16T03:03:39.171ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
75762025-12-16T03:03:39.171ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 10 reason = Timeout
75772025-12-16T03:03:39.171ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 10 reason = Timeout
75782025-12-16T03:03:39.171ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
75792025-12-16T03:03:39.171ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
75802025-12-16T03:03:39.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
75812025-12-16T03:03:39.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
75822025-12-16T03:03:39.177ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1226, tv_nsec: 791440457 } })
75832025-12-16T03:03:39.178ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1226, tv_nsec: 792483851 } })
75842025-12-16T03:03:39.178ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
75852025-12-16T03:03:39.179ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1226, tv_nsec: 793226460 } })
75862025-12-16T03:03:39.181ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 10.053468ms iteration = 10 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
75872025-12-16T03:03:39.211ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 25 reason = Timeout
75882025-12-16T03:03:39.211ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 28 reason = Timeout
75892025-12-16T03:03:39.211ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 55 reason = Timeout
75902025-12-16T03:03:39.211ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
75912025-12-16T03:03:39.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
75922025-12-16T03:03:39.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75932025-12-16T03:03:39.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
75942025-12-16T03:03:39.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
75952025-12-16T03:03:39.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
75962025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 10 reason = Timeout
75972025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
75982025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 10 reason = Timeout
75992025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 10 reason = Timeout
76002025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
76012025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
76022025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
76032025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 28 reason = Timeout
76042025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 10 reason = Timeout
76052025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
76062025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 15 reason = Timeout
76072025-12-16T03:03:39.213ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
76082025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76092025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
76102025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 10 reason = Timeout
76112025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
76122025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 10 reason = Timeout
76132025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
76142025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 17 reason = Timeout
76152025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
76162025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 15 reason = Timeout
76172025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
76182025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
76192025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
76202025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
76212025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76222025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
76232025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
76242025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
76252025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
76262025-12-16T03:03:39.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
76272025-12-16T03:03:39.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
76282025-12-16T03:03:39.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76292025-12-16T03:03:39.214ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76302025-12-16T03:03:39.215ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76312025-12-16T03:03:39.216ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76322025-12-16T03:03:39.216ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76332025-12-16T03:03:39.216ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76342025-12-16T03:03:39.216ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76352025-12-16T03:03:39.216ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
76362025-12-16T03:03:39.217ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
76372025-12-16T03:03:39.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
76382025-12-16T03:03:39.217ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
76392025-12-16T03:03:39.217ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
76402025-12-16T03:03:39.217ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
76412025-12-16T03:03:39.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
76422025-12-16T03:03:39.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
76432025-12-16T03:03:39.219ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76442025-12-16T03:03:39.221ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
76452025-12-16T03:03:39.221ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
76462025-12-16T03:03:39.221ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
76472025-12-16T03:03:39.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
76482025-12-16T03:03:39.222ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
76492025-12-16T03:03:39.222ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
76502025-12-16T03:03:39.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
76512025-12-16T03:03:39.222ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
76522025-12-16T03:03:39.222ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76532025-12-16T03:03:39.222ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
76542025-12-16T03:03:39.222ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
76552025-12-16T03:03:39.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76562025-12-16T03:03:39.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
76572025-12-16T03:03:39.223ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
76582025-12-16T03:03:39.223ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
76592025-12-16T03:03:39.223ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
76602025-12-16T03:03:39.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76612025-12-16T03:03:39.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
76622025-12-16T03:03:39.223ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
76632025-12-16T03:03:39.223ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
76642025-12-16T03:03:39.223ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
76652025-12-16T03:03:39.223ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
76662025-12-16T03:03:39.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
76672025-12-16T03:03:39.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
76682025-12-16T03:03:39.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
76692025-12-16T03:03:39.224ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = b48a1e8c-4c75-4571-b0b8-834af8e60fa2 original_time_started = 2025-12-16 03:03:11.058329 UTC
76702025-12-16T03:03:39.224ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 12.853453ms iteration = 25 status = {"Loaded":{"collection_id":"b48a1e8c-4c75-4571-b0b8-834af8e60fa2","time_loaded":"2025-12-16T03:03:39.211534590Z","time_started":"2025-12-16T03:03:11.058329Z"}}
76712025-12-16T03:03:39.225ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76722025-12-16T03:03:39.226ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
76732025-12-16T03:03:39.226ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
76742025-12-16T03:03:39.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
76752025-12-16T03:03:39.227ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1226, tv_nsec: 841938387 } })
76762025-12-16T03:03:39.228ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
76772025-12-16T03:03:39.228ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1226, tv_nsec: 842800608 } })
76782025-12-16T03:03:39.229ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
76792025-12-16T03:03:39.229ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
76802025-12-16T03:03:39.229ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76812025-12-16T03:03:39.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
76822025-12-16T03:03:39.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76832025-12-16T03:03:39.229ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
76842025-12-16T03:03:39.229ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 18.307212ms iteration = 55 status = {"config_updated":false}
76852025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 10 reason = Timeout
76862025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 10 reason = Timeout
76872025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
76882025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 16.65\u{b5}s iteration = 10 status = {"disabled":true,"errors":[],"sps":[]}
76892025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 10 reason = Timeout
76902025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 10 reason = Timeout
76912025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
76922025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 10 reason = Timeout
76932025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
76942025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
76952025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 19 reason = Timeout
76962025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 10 reason = Timeout
76972025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
76982025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
76992025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 19 reason = Timeout
77002025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
77012025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
77022025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
77032025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
77042025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
77052025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
77062025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
77072025-12-16T03:03:39.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
77082025-12-16T03:03:39.231ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
77092025-12-16T03:03:39.231ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 18.23806ms iteration = 10 status = 0
77102025-12-16T03:03:39.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
77112025-12-16T03:03:39.231ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
77122025-12-16T03:03:39.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
77132025-12-16T03:03:39.232ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77142025-12-16T03:03:39.233ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77152025-12-16T03:03:39.233ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
77162025-12-16T03:03:39.233ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 20.649961ms iteration = 10 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
77172025-12-16T03:03:39.235ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77182025-12-16T03:03:39.237ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
77192025-12-16T03:03:39.237ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
77202025-12-16T03:03:39.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
77212025-12-16T03:03:39.238ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
77222025-12-16T03:03:39.239ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
77232025-12-16T03:03:39.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
77242025-12-16T03:03:39.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
77252025-12-16T03:03:39.240ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 10.484363ms iteration = 19 status = "NoSitrep"
77262025-12-16T03:03:39.244ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
77272025-12-16T03:03:39.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
77282025-12-16T03:03:39.252ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
77292025-12-16T03:03:39.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
77302025-12-16T03:03:39.256ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77312025-12-16T03:03:39.259ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
77322025-12-16T03:03:39.261ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
77332025-12-16T03:03:39.261ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 30.854731ms iteration = 19 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
77342025-12-16T03:03:39.277ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
77352025-12-16T03:03:39.277ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 66.032927ms iteration = 28 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
77362025-12-16T03:03:39.286ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77372025-12-16T03:03:39.290ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
77382025-12-16T03:03:39.290ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
77392025-12-16T03:03:39.290ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "782976b3-74b1-4c43-99d6-69282fadb520", "content-length": "26", "date": "Tue, 16 Dec 2025 03:03:39 GMT"} })
77402025-12-16T03:03:39.290ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
77412025-12-16T03:03:39.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
77422025-12-16T03:03:39.290ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
77432025-12-16T03:03:39.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
77442025-12-16T03:03:39.292ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
77452025-12-16T03:03:39.293ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1226, tv_nsec: 907199354 } })
77462025-12-16T03:03:39.293ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
77472025-12-16T03:03:39.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
77482025-12-16T03:03:39.295ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
77492025-12-16T03:03:39.295ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77502025-12-16T03:03:39.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
77512025-12-16T03:03:39.297ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 83.990934ms iteration = 28 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
77522025-12-16T03:03:39.297ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77532025-12-16T03:03:39.298ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
77542025-12-16T03:03:39.298ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
77552025-12-16T03:03:39.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
77562025-12-16T03:03:39.300ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
77572025-12-16T03:03:39.300ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
77582025-12-16T03:03:39.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
77592025-12-16T03:03:39.300ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
77602025-12-16T03:03:39.300ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
77612025-12-16T03:03:39.300ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 87.743751ms iteration = 10 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
77622025-12-16T03:03:39.302ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
77632025-12-16T03:03:39.302ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 88.948957ms iteration = 15 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
77642025-12-16T03:03:39.302ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
77652025-12-16T03:03:39.302ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
77662025-12-16T03:03:39.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
77672025-12-16T03:03:39.304ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 132.857064ms iteration = 10 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
77682025-12-16T03:03:39.304ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 74.119588ms iteration = 10 status = {"errors":[],"finish_invoked_ok":[]}
77692025-12-16T03:03:39.304ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 91.57649ms iteration = 10 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
77702025-12-16T03:03:39.306ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 76.646511ms iteration = 10 status = {"errors":[],"garbage_collect_requested":[]}
77712025-12-16T03:03:39.306ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 93.613606ms iteration = 17 status = {"physical_disks_added":0}
77722025-12-16T03:03:39.310ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
77732025-12-16T03:03:39.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
77742025-12-16T03:03:39.312ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
77752025-12-16T03:03:39.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
77762025-12-16T03:03:39.314ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 84.576211ms iteration = 10 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
77772025-12-16T03:03:39.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
77782025-12-16T03:03:39.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
77792025-12-16T03:03:39.318ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 88.606562ms iteration = 10 status = {"errors":[],"requests_created_ok":[]}
77802025-12-16T03:03:39.336ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
77812025-12-16T03:03:39.336ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
77822025-12-16T03:03:39.337ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "43d9e227-c1f4-489b-a66a-1ed1b73dd5b9", "content-length": "372", "date": "Tue, 16 Dec 2025 03:03:39 GMT"} })
77832025-12-16T03:03:39.337ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
77842025-12-16T03:03:39.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
77852025-12-16T03:03:39.359ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
77862025-12-16T03:03:39.359ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
77872025-12-16T03:03:39.360ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "4fe86dab-6081-45fd-b049-58575c22e21e", "date": "Tue, 16 Dec 2025 03:03:39 GMT"} })
77882025-12-16T03:03:39.360ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
77892025-12-16T03:03:39.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
77902025-12-16T03:03:39.363ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
77912025-12-16T03:03:39.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
77922025-12-16T03:03:39.364ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 134.882509ms iteration = 10 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
77932025-12-16T03:03:39.366ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
77942025-12-16T03:03:39.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
77952025-12-16T03:03:39.369ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
77962025-12-16T03:03:39.369ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
77972025-12-16T03:03:39.372ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "75afd1c1-26eb-4eb8-9556-b4cb71dab1d9", "date": "Tue, 16 Dec 2025 03:03:39 GMT"} })
77982025-12-16T03:03:39.374ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 160.95935ms iteration = 15 status = {}
77992025-12-16T03:03:39.386ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
78002025-12-16T03:03:39.386ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
78012025-12-16T03:03:39.387ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "32a88db3-758d-4f5f-82b3-776c963292e2", "content-length": "26", "date": "Tue, 16 Dec 2025 03:03:39 GMT"} })
78022025-12-16T03:03:39.387ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
78032025-12-16T03:03:39.388ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
78042025-12-16T03:03:39.411ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
78052025-12-16T03:03:39.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
78062025-12-16T03:03:39.413ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78072025-12-16T03:03:39.413ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 242.639284ms iteration = 10 status = {}
78082025-12-16T03:03:39.415ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
78092025-12-16T03:03:39.415ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
78102025-12-16T03:03:39.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
78112025-12-16T03:03:39.417ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78122025-12-16T03:03:39.419ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
78132025-12-16T03:03:39.419ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
78142025-12-16T03:03:39.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
78152025-12-16T03:03:39.434ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1227, tv_nsec: 49082581 } })
78162025-12-16T03:03:39.462ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
78172025-12-16T03:03:39.463ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9334ecb-c9bc-4692-bc31-2c3bf660d55c", "content-length": "691", "date": "Tue, 16 Dec 2025 03:03:39 GMT"} })
78182025-12-16T03:03:39.463ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
78192025-12-16T03:03:39.463ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
78202025-12-16T03:03:39.463ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
78212025-12-16T03:03:39.464ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 251.219752ms iteration = 10 status = {}
78222025-12-16T03:03:39.472ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1227, tv_nsec: 86463785 } })
78232025-12-16T03:03:39.504ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78242025-12-16T03:03:39.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78252025-12-16T03:03:39.506ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
78262025-12-16T03:03:39.506ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78272025-12-16T03:03:39.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78282025-12-16T03:03:39.518ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
78292025-12-16T03:03:39.518ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
78302025-12-16T03:03:39.518ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
78312025-12-16T03:03:39.518ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
78322025-12-16T03:03:39.518ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
78332025-12-16T03:03:39.518ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
78342025-12-16T03:03:39.518ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
78352025-12-16T03:03:39.518ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
78362025-12-16T03:03:39.518ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
78372025-12-16T03:03:39.519ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
78382025-12-16T03:03:39.519ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
78392025-12-16T03:03:39.519ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78402025-12-16T03:03:39.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78412025-12-16T03:03:39.522ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
78422025-12-16T03:03:39.522ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
78432025-12-16T03:03:39.681ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "1dde2a6e-ad81-4ee5-866a-a73881f91f62", "date": "Tue, 16 Dec 2025 03:03:39 GMT"} })
78442025-12-16T03:03:39.681ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
78452025-12-16T03:03:39.682ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
78462025-12-16T03:03:39.682ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb2647c7-3dfa-421b-967f-7b008941ed0e", "content-length": "682", "date": "Tue, 16 Dec 2025 03:03:39 GMT"} })
78472025-12-16T03:03:39.682ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78482025-12-16T03:03:39.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78492025-12-16T03:03:39.686ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78502025-12-16T03:03:39.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78512025-12-16T03:03:39.689ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78522025-12-16T03:03:39.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78532025-12-16T03:03:39.691ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78542025-12-16T03:03:39.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78552025-12-16T03:03:39.699ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 485.758034ms iteration = 10 status = {}
78562025-12-16T03:03:41.063ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 495 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = ec7f16ee-3205-47c1-9974-7aedffc4bcb6 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
78572025-12-16T03:03:44.212ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 56 reason = Timeout
78582025-12-16T03:03:44.212ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
78592025-12-16T03:03:44.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
78602025-12-16T03:03:44.213ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78612025-12-16T03:03:44.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
78622025-12-16T03:03:44.216ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
78632025-12-16T03:03:44.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
78642025-12-16T03:03:44.218ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
78652025-12-16T03:03:44.218ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.744345ms iteration = 56 status = {"config_updated":false}
78662025-12-16T03:03:49.211ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 29 reason = Timeout
78672025-12-16T03:03:49.211ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 57 reason = Timeout
78682025-12-16T03:03:49.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
78692025-12-16T03:03:49.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
78702025-12-16T03:03:49.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
78712025-12-16T03:03:49.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
78722025-12-16T03:03:49.213ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 29 reason = Timeout
78732025-12-16T03:03:49.213ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
78742025-12-16T03:03:49.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
78752025-12-16T03:03:49.213ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78762025-12-16T03:03:49.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78772025-12-16T03:03:49.214ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
78782025-12-16T03:03:49.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
78792025-12-16T03:03:49.216ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.300632ms iteration = 29 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
78802025-12-16T03:03:49.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
78812025-12-16T03:03:49.217ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
78822025-12-16T03:03:49.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
78832025-12-16T03:03:49.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
78842025-12-16T03:03:49.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
78852025-12-16T03:03:49.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
78862025-12-16T03:03:49.219ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
78872025-12-16T03:03:49.219ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.158211ms iteration = 57 status = {"config_updated":false}
78882025-12-16T03:03:49.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
78892025-12-16T03:03:49.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
78902025-12-16T03:03:49.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78912025-12-16T03:03:49.223ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
78922025-12-16T03:03:49.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
78932025-12-16T03:03:49.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
78942025-12-16T03:03:49.235ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
78952025-12-16T03:03:49.235ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.334046ms iteration = 29 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
78962025-12-16T03:03:51.063ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 271 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 257379c8-d3dc-431f-95d1-05f4a411bc1b response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
78972025-12-16T03:03:54.211ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 26 reason = Timeout
78982025-12-16T03:03:54.211ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 58 reason = Timeout
78992025-12-16T03:03:54.211ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
79002025-12-16T03:03:54.211ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
79012025-12-16T03:03:54.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
79022025-12-16T03:03:54.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
79032025-12-16T03:03:54.214ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79042025-12-16T03:03:54.214ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79052025-12-16T03:03:54.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
79062025-12-16T03:03:54.218ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
79072025-12-16T03:03:54.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
79082025-12-16T03:03:54.218ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
79092025-12-16T03:03:54.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
79102025-12-16T03:03:54.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
79112025-12-16T03:03:54.219ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = b48a1e8c-4c75-4571-b0b8-834af8e60fa2 original_time_started = 2025-12-16 03:03:11.058329 UTC
79122025-12-16T03:03:54.219ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.634967ms iteration = 26 status = {"Loaded":{"collection_id":"b48a1e8c-4c75-4571-b0b8-834af8e60fa2","time_loaded":"2025-12-16T03:03:54.211771700Z","time_started":"2025-12-16T03:03:11.058329Z"}}
79132025-12-16T03:03:54.220ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
79142025-12-16T03:03:54.220ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.541478ms iteration = 58 status = {"config_updated":false}
79152025-12-16T03:03:54.230ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 20 reason = Timeout
79162025-12-16T03:03:54.230ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 20 reason = Timeout
79172025-12-16T03:03:54.230ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
79182025-12-16T03:03:54.230ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
79192025-12-16T03:03:54.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
79202025-12-16T03:03:54.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
79212025-12-16T03:03:54.231ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
79222025-12-16T03:03:54.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
79232025-12-16T03:03:54.231ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79242025-12-16T03:03:54.232ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79252025-12-16T03:03:54.234ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
79262025-12-16T03:03:54.234ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
79272025-12-16T03:03:54.234ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
79282025-12-16T03:03:54.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
79292025-12-16T03:03:54.235ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.349617ms iteration = 20 status = "NoSitrep"
79302025-12-16T03:03:54.237ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
79312025-12-16T03:03:54.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
79322025-12-16T03:03:54.237ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
79332025-12-16T03:03:54.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
79342025-12-16T03:03:54.239ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79352025-12-16T03:03:54.240ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
79362025-12-16T03:03:54.241ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
79372025-12-16T03:03:54.241ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.874838ms iteration = 20 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
79382025-12-16T03:03:59.212ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 30 reason = Timeout
79392025-12-16T03:03:59.212ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 59 reason = Timeout
79402025-12-16T03:03:59.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79412025-12-16T03:03:59.212ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
79422025-12-16T03:03:59.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
79432025-12-16T03:03:59.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
79442025-12-16T03:03:59.213ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 30 reason = Timeout
79452025-12-16T03:03:59.213ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
79462025-12-16T03:03:59.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
79472025-12-16T03:03:59.214ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79482025-12-16T03:03:59.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79492025-12-16T03:03:59.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
79502025-12-16T03:03:59.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
79512025-12-16T03:03:59.218ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
79522025-12-16T03:03:59.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79532025-12-16T03:03:59.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
79542025-12-16T03:03:59.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
79552025-12-16T03:03:59.220ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
79562025-12-16T03:03:59.220ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.409173ms iteration = 59 status = {"config_updated":false}
79572025-12-16T03:03:59.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79582025-12-16T03:03:59.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
79592025-12-16T03:03:59.221ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79602025-12-16T03:03:59.224ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
79612025-12-16T03:03:59.224ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79622025-12-16T03:03:59.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
79632025-12-16T03:03:59.236ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
79642025-12-16T03:03:59.236ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.176136ms iteration = 30 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
79652025-12-16T03:03:59.249ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
79662025-12-16T03:03:59.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
79672025-12-16T03:03:59.251ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.974031ms iteration = 30 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
79682025-12-16T03:04:01.063ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 299 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = d2352366-87d7-4662-a1bf-9bd9808faf3f response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
79692025-12-16T03:04:04.212ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 60 reason = Timeout
79702025-12-16T03:04:04.212ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
79712025-12-16T03:04:04.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
79722025-12-16T03:04:04.214ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79732025-12-16T03:04:04.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
79742025-12-16T03:04:04.217ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
79752025-12-16T03:04:04.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
79762025-12-16T03:04:04.219ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
79772025-12-16T03:04:04.219ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.933467ms iteration = 60 status = {"config_updated":false}
79782025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 6 reason = Timeout
79792025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 6 reason = Timeout
79802025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
79812025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
79822025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 6 reason = Timeout
79832025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
79842025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
79852025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 8 reason = Timeout
79862025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 7 reason = Timeout
79872025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 6 reason = Timeout
79882025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
79892025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 6 reason = Timeout
79902025-12-16T03:04:09.171ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
79912025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 11 reason = Timeout
79922025-12-16T03:04:09.171ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-12-16 02:54:09.171386125 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
79932025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
79942025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
79952025-12-16T03:04:09.171ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b3d05c1-ce7f-403d-9280-dcc4f31baae8 step component = ExternalNetworking step id = Ensure
79962025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 11 reason = Timeout
79972025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
79982025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
79992025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 6 reason = Timeout
80002025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 8 reason = Timeout
80012025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
80022025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
80032025-12-16T03:04:09.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
80042025-12-16T03:04:09.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
80052025-12-16T03:04:09.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
80062025-12-16T03:04:09.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
80072025-12-16T03:04:09.172ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1a45cd23-3d0a-43eb-adce-62c7b712baf1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1256, tv_nsec: 785710545 } })
80082025-12-16T03:04:09.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
80092025-12-16T03:04:09.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
80102025-12-16T03:04:09.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
80112025-12-16T03:04:09.172ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("49a51a59-9919-48ae-8182-4070a8430160.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6af60386-8771-4401-9d4d-e93774e519f0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 1256, tv_nsec: 785855357 } })
80122025-12-16T03:04:09.172ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
80132025-12-16T03:04:09.172ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.422098ms iteration = 6 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
80142025-12-16T03:04:09.172ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
80152025-12-16T03:04:09.172ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.422098ms iteration = 6 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
80162025-12-16T03:04:09.173ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
80172025-12-16T03:04:09.173ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
80182025-12-16T03:04:09.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
80192025-12-16T03:04:09.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
80202025-12-16T03:04:09.174ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80212025-12-16T03:04:09.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
80222025-12-16T03:04:09.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
80232025-12-16T03:04:09.175ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-12-16 02:54:09.171386125 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
80242025-12-16T03:04:09.175ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.104682ms iteration = 6 status = {"expiration":"2025-12-16T02:54:09.171386125Z","pruned":[]}
80252025-12-16T03:04:09.175ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1256, tv_nsec: 789166019 } })
80262025-12-16T03:04:09.175ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80272025-12-16T03:04:09.175ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1256, tv_nsec: 789274880 } })
80282025-12-16T03:04:09.175ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
80292025-12-16T03:04:09.176ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1256, tv_nsec: 790060700 } })
80302025-12-16T03:04:09.176ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80312025-12-16T03:04:09.178ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
80322025-12-16T03:04:09.179ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
80332025-12-16T03:04:09.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
80342025-12-16T03:04:09.181ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 10.158088ms iteration = 11 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
80352025-12-16T03:04:09.181ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
80362025-12-16T03:04:09.182ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
80372025-12-16T03:04:09.186ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
80382025-12-16T03:04:09.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
80392025-12-16T03:04:09.188ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
80402025-12-16T03:04:09.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
80412025-12-16T03:04:09.189ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80422025-12-16T03:04:09.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80432025-12-16T03:04:09.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
80442025-12-16T03:04:09.196ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
80452025-12-16T03:04:09.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
80462025-12-16T03:04:09.199ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
80472025-12-16T03:04:09.199ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
80482025-12-16T03:04:09.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
80492025-12-16T03:04:09.202ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80502025-12-16T03:04:09.206ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
80512025-12-16T03:04:09.206ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
80522025-12-16T03:04:09.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
80532025-12-16T03:04:09.210ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
80542025-12-16T03:04:09.212ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 15 reason = Timeout
80552025-12-16T03:04:09.212ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 31 reason = Timeout
80562025-12-16T03:04:09.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
80572025-12-16T03:04:09.212ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 27 reason = Timeout
80582025-12-16T03:04:09.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
80592025-12-16T03:04:09.212ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
80602025-12-16T03:04:09.212ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 61 reason = Timeout
80612025-12-16T03:04:09.212ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
80622025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
80632025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
80642025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
80652025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
80662025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 9 reason = Timeout
80672025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 11 reason = Timeout
80682025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 16 reason = Timeout
80692025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 6 reason = Timeout
80702025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 11 reason = Timeout
80712025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 2 reason = Timeout
80722025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 6 reason = Timeout
80732025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 11 reason = Timeout
80742025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 11 reason = Timeout
80752025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 31 reason = Timeout
80762025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 11 reason = Timeout
80772025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 18 reason = Timeout
80782025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 11 reason = Timeout
80792025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 16 reason = Timeout
80802025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 6 reason = Timeout
80812025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
80822025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
80832025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
80842025-12-16T03:04:09.213ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
80852025-12-16T03:04:09.214ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
80862025-12-16T03:04:09.214ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
80872025-12-16T03:04:09.214ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
80882025-12-16T03:04:09.214ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
80892025-12-16T03:04:09.214ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
80902025-12-16T03:04:09.214ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
80912025-12-16T03:04:09.214ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
80922025-12-16T03:04:09.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
80932025-12-16T03:04:09.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
80942025-12-16T03:04:09.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
80952025-12-16T03:04:09.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
80962025-12-16T03:04:09.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
80972025-12-16T03:04:09.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
80982025-12-16T03:04:09.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
80992025-12-16T03:04:09.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
81002025-12-16T03:04:09.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
81012025-12-16T03:04:09.214ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
81022025-12-16T03:04:09.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
81032025-12-16T03:04:09.215ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
81042025-12-16T03:04:09.215ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
81052025-12-16T03:04:09.215ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
81062025-12-16T03:04:09.215ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
81072025-12-16T03:04:09.215ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
81082025-12-16T03:04:09.215ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
81092025-12-16T03:04:09.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
81102025-12-16T03:04:09.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
81112025-12-16T03:04:09.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
81122025-12-16T03:04:09.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
81132025-12-16T03:04:09.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
81142025-12-16T03:04:09.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
81152025-12-16T03:04:09.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81162025-12-16T03:04:09.216ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
81172025-12-16T03:04:09.218ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81182025-12-16T03:04:09.218ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81192025-12-16T03:04:09.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81202025-12-16T03:04:09.218ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81212025-12-16T03:04:09.221ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
81222025-12-16T03:04:09.221ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81232025-12-16T03:04:09.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81242025-12-16T03:04:09.222ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81252025-12-16T03:04:09.223ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
81262025-12-16T03:04:09.223ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
81272025-12-16T03:04:09.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
81282025-12-16T03:04:09.224ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
81292025-12-16T03:04:09.224ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
81302025-12-16T03:04:09.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
81312025-12-16T03:04:09.225ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
81322025-12-16T03:04:09.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81332025-12-16T03:04:09.225ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81342025-12-16T03:04:09.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
81352025-12-16T03:04:09.226ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81362025-12-16T03:04:09.226ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
81372025-12-16T03:04:09.226ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
81382025-12-16T03:04:09.226ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81392025-12-16T03:04:09.226ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81402025-12-16T03:04:09.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
81412025-12-16T03:04:09.228ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81422025-12-16T03:04:09.228ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81432025-12-16T03:04:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
81442025-12-16T03:04:09.228ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
81452025-12-16T03:04:09.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
81462025-12-16T03:04:09.229ZDEBGnexus (BackgroundTasks): activating background_task = tuf_repo_pruner iteration = 3 reason = Timeout
81472025-12-16T03:04:09.229ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 11 reason = Timeout
81482025-12-16T03:04:09.229ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
81492025-12-16T03:04:09.229ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 16.05\u{b5}s iteration = 11 status = {"disabled":true,"errors":[],"sps":[]}
81502025-12-16T03:04:09.229ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 6 reason = Timeout
81512025-12-16T03:04:09.229ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 6 reason = Timeout
81522025-12-16T03:04:09.229ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 6 reason = Timeout
81532025-12-16T03:04:09.229ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
81542025-12-16T03:04:09.229ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
81552025-12-16T03:04:09.229ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
81562025-12-16T03:04:09.229ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
81572025-12-16T03:04:09.229ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 19 reason = Timeout
81582025-12-16T03:04:09.229ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
81592025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 2 reason = Timeout
81602025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 19.051\u{b5}s iteration = 19 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
81612025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
81622025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 6 reason = Timeout
81632025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 11 reason = Timeout
81642025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
81652025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
81662025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 11 reason = Timeout
81672025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
81682025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 11 reason = Timeout
81692025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
81702025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 11 reason = Timeout
81712025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
81722025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 21 reason = Timeout
81732025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
81742025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 11 reason = Timeout
81752025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
81762025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 21 reason = Timeout
81772025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
81782025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
81792025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
81802025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81812025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
81822025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
81832025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
81842025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
81852025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
81862025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
81872025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
81882025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
81892025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
81902025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
81912025-12-16T03:04:09.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
81922025-12-16T03:04:09.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
81932025-12-16T03:04:09.231ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = b48a1e8c-4c75-4571-b0b8-834af8e60fa2 original_time_started = 2025-12-16 03:03:11.058329 UTC
81942025-12-16T03:04:09.231ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 18.88729ms iteration = 27 status = {"Loaded":{"collection_id":"b48a1e8c-4c75-4571-b0b8-834af8e60fa2","time_loaded":"2025-12-16T03:04:09.212539136Z","time_started":"2025-12-16T03:03:11.058329Z"}}
81952025-12-16T03:04:09.232ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
81962025-12-16T03:04:09.232ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
81972025-12-16T03:04:09.232ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81982025-12-16T03:04:09.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
81992025-12-16T03:04:09.232ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
82002025-12-16T03:04:09.232ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
82012025-12-16T03:04:09.232ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
82022025-12-16T03:04:09.232ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
82032025-12-16T03:04:09.232ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
82042025-12-16T03:04:09.232ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
82052025-12-16T03:04:09.232ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
82062025-12-16T03:04:09.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
82072025-12-16T03:04:09.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
82082025-12-16T03:04:09.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
82092025-12-16T03:04:09.234ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
82102025-12-16T03:04:09.234ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
82112025-12-16T03:04:09.234ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
82122025-12-16T03:04:09.234ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
82132025-12-16T03:04:09.234ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
82142025-12-16T03:04:09.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82152025-12-16T03:04:09.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
82162025-12-16T03:04:09.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
82172025-12-16T03:04:09.235ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
82182025-12-16T03:04:09.235ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 22.792029ms iteration = 61 status = {"config_updated":false}
82192025-12-16T03:04:09.235ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
82202025-12-16T03:04:09.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
82212025-12-16T03:04:09.236ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
82222025-12-16T03:04:09.236ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
82232025-12-16T03:04:09.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
82242025-12-16T03:04:09.237ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
82252025-12-16T03:04:09.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
82262025-12-16T03:04:09.237ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
82272025-12-16T03:04:09.237ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 7.712408ms iteration = 6 status = {"by_rx":{},"error":null}
82282025-12-16T03:04:09.238ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
82292025-12-16T03:04:09.240ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
82302025-12-16T03:04:09.242ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82312025-12-16T03:04:09.243ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
82322025-12-16T03:04:09.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82332025-12-16T03:04:09.247ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Fleet result = Ok(())
82342025-12-16T03:04:09.247ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
82352025-12-16T03:04:09.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
82362025-12-16T03:04:09.251ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
82372025-12-16T03:04:09.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
82382025-12-16T03:04:09.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
82392025-12-16T03:04:09.256ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
82402025-12-16T03:04:09.256ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
82412025-12-16T03:04:09.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
82422025-12-16T03:04:09.256ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82432025-12-16T03:04:09.259ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82442025-12-16T03:04:09.260ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
82452025-12-16T03:04:09.260ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
82462025-12-16T03:04:09.260ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
82472025-12-16T03:04:09.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
82482025-12-16T03:04:09.261ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
82492025-12-16T03:04:09.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
82502025-12-16T03:04:09.262ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 33.014508ms iteration = 6 status = {"errors":[],"found_port_ok":[]}
82512025-12-16T03:04:09.263ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
82522025-12-16T03:04:09.263ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
82532025-12-16T03:04:09.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
82542025-12-16T03:04:09.265ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
82552025-12-16T03:04:09.268ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
82562025-12-16T03:04:09.271ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
82572025-12-16T03:04:09.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
82582025-12-16T03:04:09.277ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
82592025-12-16T03:04:09.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
82602025-12-16T03:04:09.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
82612025-12-16T03:04:09.287ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
82622025-12-16T03:04:09.288ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b3d05c1-ce7f-403d-9280-dcc4f31baae8 step component = SledList step id = Fetch
82632025-12-16T03:04:09.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
82642025-12-16T03:04:09.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
82652025-12-16T03:04:09.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82662025-12-16T03:04:09.298ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
82672025-12-16T03:04:09.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
82682025-12-16T03:04:09.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
82692025-12-16T03:04:09.303ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
82702025-12-16T03:04:09.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
82712025-12-16T03:04:09.307ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
82722025-12-16T03:04:09.307ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
82732025-12-16T03:04:09.307ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
82742025-12-16T03:04:09.307ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
82752025-12-16T03:04:09.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
82762025-12-16T03:04:09.311ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82772025-12-16T03:04:09.311ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82782025-12-16T03:04:09.314ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
82792025-12-16T03:04:09.315ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82802025-12-16T03:04:09.315ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
82812025-12-16T03:04:09.316ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
82822025-12-16T03:04:09.316ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
82832025-12-16T03:04:09.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
82842025-12-16T03:04:09.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
82852025-12-16T03:04:09.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
82862025-12-16T03:04:09.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
82872025-12-16T03:04:09.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
82882025-12-16T03:04:09.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
82892025-12-16T03:04:09.323ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 93.375433ms iteration = 21 status = "NoSitrep"
82902025-12-16T03:04:09.324ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 94.6584ms iteration = 6 status = {"dispatched":[],"errors":[],"glob_version":"212.0.0","globs_reprocessed":{},"no_receivers":[]}
82912025-12-16T03:04:09.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
82922025-12-16T03:04:09.324ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
82932025-12-16T03:04:09.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
82942025-12-16T03:04:09.326ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
82952025-12-16T03:04:09.326ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
82962025-12-16T03:04:09.326ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
82972025-12-16T03:04:09.326ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
82982025-12-16T03:04:09.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
82992025-12-16T03:04:09.327ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83002025-12-16T03:04:09.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83012025-12-16T03:04:09.328ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-12-16T02:59:05.020237Z, time_modified: 2025-12-16T02:59:05.020237Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }) }
83022025-12-16T03:04:09.328ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 157.661597ms iteration = 6 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]}},"default_endpoint":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
83032025-12-16T03:04:09.329ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
83042025-12-16T03:04:09.332ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
83052025-12-16T03:04:09.332ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
83062025-12-16T03:04:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
83072025-12-16T03:04:09.332ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
83082025-12-16T03:04:09.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
83092025-12-16T03:04:09.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
83102025-12-16T03:04:09.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83112025-12-16T03:04:09.334ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83122025-12-16T03:04:09.335ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
83132025-12-16T03:04:09.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83142025-12-16T03:04:09.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
83152025-12-16T03:04:09.335ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83162025-12-16T03:04:09.337ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
83172025-12-16T03:04:09.337ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
83182025-12-16T03:04:09.337ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83192025-12-16T03:04:09.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
83202025-12-16T03:04:09.338ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
83212025-12-16T03:04:09.338ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 125.448839ms iteration = 31 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
83222025-12-16T03:04:09.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
83232025-12-16T03:04:09.339ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
83242025-12-16T03:04:09.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
83252025-12-16T03:04:09.341ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
83262025-12-16T03:04:09.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
83272025-12-16T03:04:09.341ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 128.367477ms iteration = 11 status = 0
83282025-12-16T03:04:09.342ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83292025-12-16T03:04:09.342ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
83302025-12-16T03:04:09.342ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
83312025-12-16T03:04:09.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
83322025-12-16T03:04:09.344ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83332025-12-16T03:04:09.345ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83342025-12-16T03:04:09.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83352025-12-16T03:04:09.347ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
83362025-12-16T03:04:09.347ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 134.700777ms iteration = 31 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
83372025-12-16T03:04:09.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83382025-12-16T03:04:09.348ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83392025-12-16T03:04:09.348ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83402025-12-16T03:04:09.350ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
83412025-12-16T03:04:09.350ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 136.546711ms iteration = 11 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
83422025-12-16T03:04:09.350ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
83432025-12-16T03:04:09.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
83442025-12-16T03:04:09.350ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
83452025-12-16T03:04:09.350ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
83462025-12-16T03:04:09.350ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83472025-12-16T03:04:09.350ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Fleet result = Ok(())
83482025-12-16T03:04:09.350ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
83492025-12-16T03:04:09.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
83502025-12-16T03:04:09.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
83512025-12-16T03:04:09.350ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83522025-12-16T03:04:09.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
83532025-12-16T03:04:09.351ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83542025-12-16T03:04:09.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83552025-12-16T03:04:09.353ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
83562025-12-16T03:04:09.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
83572025-12-16T03:04:09.353ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
83582025-12-16T03:04:09.354ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
83592025-12-16T03:04:09.354ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
83602025-12-16T03:04:09.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
83612025-12-16T03:04:09.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
83622025-12-16T03:04:09.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
83632025-12-16T03:04:09.354ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
83642025-12-16T03:04:09.354ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
83652025-12-16T03:04:09.354ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
83662025-12-16T03:04:09.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
83672025-12-16T03:04:09.355ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83682025-12-16T03:04:09.355ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
83692025-12-16T03:04:09.355ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 125.53681ms iteration = 21 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
83702025-12-16T03:04:09.355ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
83712025-12-16T03:04:09.355ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
83722025-12-16T03:04:09.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
83732025-12-16T03:04:09.356ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
83742025-12-16T03:04:09.356ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
83752025-12-16T03:04:09.356ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
83762025-12-16T03:04:09.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
83772025-12-16T03:04:09.357ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83782025-12-16T03:04:09.359ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1256, tv_nsec: 972716594 } })
83792025-12-16T03:04:09.359ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
83802025-12-16T03:04:09.360ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
83812025-12-16T03:04:09.360ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
83822025-12-16T03:04:09.360ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1256, tv_nsec: 973754387 } })
83832025-12-16T03:04:09.362ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "eabcd862-da8d-49a6-b068-1059c49ece0a", "content-length": "26", "date": "Tue, 16 Dec 2025 03:04:09 GMT"} })
83842025-12-16T03:04:09.362ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83852025-12-16T03:04:09.362ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
83862025-12-16T03:04:09.363ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
83872025-12-16T03:04:09.363ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
83882025-12-16T03:04:09.363ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 150.348245ms iteration = 6 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
83892025-12-16T03:04:09.364ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 134.055729ms iteration = 11 status = {"errors":[],"requests_created_ok":[]}
83902025-12-16T03:04:09.364ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
83912025-12-16T03:04:09.364ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = TargetReleaseConfig result = Ok(())
83922025-12-16T03:04:09.364ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
83932025-12-16T03:04:09.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
83942025-12-16T03:04:09.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
83952025-12-16T03:04:09.366ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
83962025-12-16T03:04:09.366ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
83972025-12-16T03:04:09.366ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
83982025-12-16T03:04:09.366ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
83992025-12-16T03:04:09.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
84002025-12-16T03:04:09.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
84012025-12-16T03:04:09.366ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1256, tv_nsec: 980492033 } })
84022025-12-16T03:04:09.368ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
84032025-12-16T03:04:09.377ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
84042025-12-16T03:04:09.377ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
84052025-12-16T03:04:09.378ZINFOnexus (BackgroundTasks): tuf_repo_prune decision background_task = tuf_repo_pruner file = nexus/src/app/background/tasks/tuf_repo_pruner.rs:153 status = TufRepoPrunerDetails { nkeep_recent_releases: 3, nkeep_recent_uploads: 2, repos_keep_target_release: {}, repos_keep_recent_uploads: {}, repo_prune: None, other_repos_eligible_to_prune: {}, warnings: [] }
84062025-12-16T03:04:09.378ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_repo_pruner elapsed = 148.515532ms iteration = 3 status = {"nkeep_recent_releases":3,"nkeep_recent_uploads":2,"other_repos_eligible_to_prune":[],"repo_prune":null,"repos_keep_recent_uploads":[],"repos_keep_target_release":[],"status":"enabled","warnings":[]}
84072025-12-16T03:04:09.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
84082025-12-16T03:04:09.380ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84092025-12-16T03:04:09.381ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
84102025-12-16T03:04:09.381ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84112025-12-16T03:04:09.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84122025-12-16T03:04:09.385ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
84132025-12-16T03:04:09.385ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
84142025-12-16T03:04:09.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
84152025-12-16T03:04:09.386ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84162025-12-16T03:04:09.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84172025-12-16T03:04:09.387ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 216.150569ms iteration = 11 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
84182025-12-16T03:04:09.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84192025-12-16T03:04:09.389ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84202025-12-16T03:04:09.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84212025-12-16T03:04:09.389ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 159.684684ms iteration = 11 status = {"errors":[],"garbage_collect_requested":[]}
84222025-12-16T03:04:09.391ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84232025-12-16T03:04:09.392ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
84242025-12-16T03:04:09.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
84252025-12-16T03:04:09.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
84262025-12-16T03:04:09.397ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
84272025-12-16T03:04:09.397ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 183.738108ms iteration = 9 status = {"blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","inventory_collection_id":"b48a1e8c-4c75-4571-b0b8-834af8e60fa2","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
84282025-12-16T03:04:09.397ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 167.814276ms iteration = 11 status = {"errors":[],"finish_invoked_ok":[]}
84292025-12-16T03:04:09.399ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
84302025-12-16T03:04:09.399ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
84312025-12-16T03:04:09.399ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 186.094608ms iteration = 6 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
84322025-12-16T03:04:09.401ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
84332025-12-16T03:04:09.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
84342025-12-16T03:04:09.405ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
84352025-12-16T03:04:09.405ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84362025-12-16T03:04:09.406ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84372025-12-16T03:04:09.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84382025-12-16T03:04:09.410ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
84392025-12-16T03:04:09.410ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
84402025-12-16T03:04:09.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
84412025-12-16T03:04:09.412ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84422025-12-16T03:04:09.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84432025-12-16T03:04:09.412ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
84442025-12-16T03:04:09.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
84452025-12-16T03:04:09.415ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84462025-12-16T03:04:09.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84472025-12-16T03:04:09.418ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84482025-12-16T03:04:09.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84492025-12-16T03:04:09.421ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84502025-12-16T03:04:09.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84512025-12-16T03:04:09.424ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84522025-12-16T03:04:09.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84532025-12-16T03:04:09.427ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84542025-12-16T03:04:09.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84552025-12-16T03:04:09.429ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84562025-12-16T03:04:09.432ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 202.325203ms iteration = 11 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
84572025-12-16T03:04:09.433ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84582025-12-16T03:04:09.437ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
84592025-12-16T03:04:09.437ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
84602025-12-16T03:04:09.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
84612025-12-16T03:04:09.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
84622025-12-16T03:04:09.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
84632025-12-16T03:04:09.438ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
84642025-12-16T03:04:09.438ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
84652025-12-16T03:04:09.438ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
84662025-12-16T03:04:09.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
84672025-12-16T03:04:09.440ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
84682025-12-16T03:04:09.440ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84692025-12-16T03:04:09.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84702025-12-16T03:04:09.440ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
84712025-12-16T03:04:09.440ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
84722025-12-16T03:04:09.440ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
84732025-12-16T03:04:09.440ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
84742025-12-16T03:04:09.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
84752025-12-16T03:04:09.441ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84762025-12-16T03:04:09.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
84772025-12-16T03:04:09.442ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84782025-12-16T03:04:09.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84792025-12-16T03:04:09.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84802025-12-16T03:04:09.443ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84812025-12-16T03:04:09.445ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84822025-12-16T03:04:09.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84832025-12-16T03:04:09.447ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
84842025-12-16T03:04:09.447ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
84852025-12-16T03:04:09.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
84862025-12-16T03:04:09.448ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84872025-12-16T03:04:09.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84882025-12-16T03:04:09.450ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
84892025-12-16T03:04:09.450ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
84902025-12-16T03:04:09.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
84912025-12-16T03:04:09.453ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
84922025-12-16T03:04:09.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
84932025-12-16T03:04:09.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
84942025-12-16T03:04:09.455ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84952025-12-16T03:04:09.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84962025-12-16T03:04:09.456ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
84972025-12-16T03:04:09.456ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
84982025-12-16T03:04:09.456ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
84992025-12-16T03:04:09.456ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
85002025-12-16T03:04:09.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
85012025-12-16T03:04:09.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85022025-12-16T03:04:09.464ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
85032025-12-16T03:04:09.464ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
85042025-12-16T03:04:09.464ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-12-16 02:59:18.154440 UTC dns_group = external generation = 2
85052025-12-16T03:04:09.464ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 251.072761ms iteration = 11 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
85062025-12-16T03:04:09.464ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 292.674898ms iteration = 6 status = {"generation":2}
85072025-12-16T03:04:09.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85082025-12-16T03:04:09.467ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85092025-12-16T03:04:09.468ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
85102025-12-16T03:04:09.468ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
85112025-12-16T03:04:09.468ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-12-16 02:59:16.791641 UTC dns_group = internal generation = 1
85122025-12-16T03:04:09.468ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 296.84382ms iteration = 6 status = {"generation":1}
85132025-12-16T03:04:09.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85142025-12-16T03:04:09.469ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85152025-12-16T03:04:09.472ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85162025-12-16T03:04:09.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85172025-12-16T03:04:09.475ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
85182025-12-16T03:04:09.475ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85192025-12-16T03:04:09.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85202025-12-16T03:04:09.476ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85212025-12-16T03:04:09.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85222025-12-16T03:04:09.483ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
85232025-12-16T03:04:09.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85242025-12-16T03:04:09.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85252025-12-16T03:04:09.485ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
85262025-12-16T03:04:09.485ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85272025-12-16T03:04:09.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85282025-12-16T03:04:09.487ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85292025-12-16T03:04:09.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85302025-12-16T03:04:09.488ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 258.156531ms iteration = 11 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
85312025-12-16T03:04:09.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85322025-12-16T03:04:09.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85332025-12-16T03:04:09.490ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85342025-12-16T03:04:09.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85352025-12-16T03:04:09.491ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85362025-12-16T03:04:09.491ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
85372025-12-16T03:04:09.491ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 278.218005ms iteration = 16 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
85382025-12-16T03:04:09.493ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85392025-12-16T03:04:09.496ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
85402025-12-16T03:04:09.496ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85412025-12-16T03:04:09.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85422025-12-16T03:04:09.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85432025-12-16T03:04:09.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85442025-12-16T03:04:09.502ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
85452025-12-16T03:04:09.502ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85462025-12-16T03:04:09.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85472025-12-16T03:04:09.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85482025-12-16T03:04:09.504ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
85492025-12-16T03:04:09.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
85502025-12-16T03:04:09.508ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85512025-12-16T03:04:09.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85522025-12-16T03:04:09.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
85532025-12-16T03:04:09.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85542025-12-16T03:04:09.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85552025-12-16T03:04:09.511ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85562025-12-16T03:04:09.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85572025-12-16T03:04:09.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85582025-12-16T03:04:09.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85592025-12-16T03:04:09.514ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85602025-12-16T03:04:09.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85612025-12-16T03:04:09.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85622025-12-16T03:04:09.521ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
85632025-12-16T03:04:09.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85642025-12-16T03:04:09.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85652025-12-16T03:04:09.522ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85662025-12-16T03:04:09.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85672025-12-16T03:04:09.527ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85682025-12-16T03:04:09.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85692025-12-16T03:04:09.530ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85702025-12-16T03:04:09.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85712025-12-16T03:04:09.533ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85722025-12-16T03:04:09.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85732025-12-16T03:04:09.535ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85742025-12-16T03:04:09.543ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
85752025-12-16T03:04:09.543ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85762025-12-16T03:04:09.543ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
85772025-12-16T03:04:09.543ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
85782025-12-16T03:04:09.543ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 330.157503ms iteration = 11 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
85792025-12-16T03:04:09.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85802025-12-16T03:04:09.546ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85812025-12-16T03:04:09.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85822025-12-16T03:04:09.550ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85832025-12-16T03:04:09.551ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85842025-12-16T03:04:09.552ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
85852025-12-16T03:04:09.552ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "8577e6cd-ee87-4fbb-bec2-b617d63524d7", "date": "Tue, 16 Dec 2025 03:04:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
85862025-12-16T03:04:09.552ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
85872025-12-16T03:04:09.552ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b3d05c1-ce7f-403d-9280-dcc4f31baae8 step component = DeployNexusRecords step id = Ensure
85882025-12-16T03:04:09.553ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
85892025-12-16T03:04:09.553ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85902025-12-16T03:04:09.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
85912025-12-16T03:04:09.555ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "78930221-eacd-4a2e-89c5-f3e62781886a", "date": "Tue, 16 Dec 2025 03:04:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
85922025-12-16T03:04:09.555ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
85932025-12-16T03:04:09.556ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85942025-12-16T03:04:09.556ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85952025-12-16T03:04:09.557ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85962025-12-16T03:04:09.558ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
85972025-12-16T03:04:09.558ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 387.018803ms iteration = 8 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
85982025-12-16T03:04:09.559ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
85992025-12-16T03:04:09.559ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
86002025-12-16T03:04:09.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
86012025-12-16T03:04:09.559ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86022025-12-16T03:04:09.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86032025-12-16T03:04:09.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
86042025-12-16T03:04:09.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
86052025-12-16T03:04:09.562ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
86062025-12-16T03:04:09.562ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
86072025-12-16T03:04:09.562ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86082025-12-16T03:04:09.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
86092025-12-16T03:04:09.563ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86102025-12-16T03:04:09.567ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
86112025-12-16T03:04:09.568ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
86122025-12-16T03:04:09.568ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86132025-12-16T03:04:09.568ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86142025-12-16T03:04:09.570ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86152025-12-16T03:04:09.571ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], Name("nexus"): [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
86162025-12-16T03:04:09.571ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
86172025-12-16T03:04:09.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
86182025-12-16T03:04:09.573ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86192025-12-16T03:04:09.576ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
86202025-12-16T03:04:09.576ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86212025-12-16T03:04:09.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86222025-12-16T03:04:09.577ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
86232025-12-16T03:04:09.577ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
86242025-12-16T03:04:09.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
86252025-12-16T03:04:09.579ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
86262025-12-16T03:04:09.579ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
86272025-12-16T03:04:09.580ZDEBGnexus (service-firewall-rule-progator): resolved 3 rules for sleds background_task = service_firewall_rule_propagation
86282025-12-16T03:04:09.582ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86292025-12-16T03:04:09.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86302025-12-16T03:04:09.588ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86312025-12-16T03:04:09.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86322025-12-16T03:04:09.589ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
86332025-12-16T03:04:09.589ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "8b3edb49-e776-4cec-80b6-4ea823d656a5", "date": "Tue, 16 Dec 2025 03:04:09 GMT"} })
86342025-12-16T03:04:09.589ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
86352025-12-16T03:04:09.591ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86362025-12-16T03:04:09.592ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
86372025-12-16T03:04:09.592ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 362.745086ms iteration = 6 status = {"errors":[],"probes_by_sled":{"b1c76b61-acee-4be6-b582-973c938d6b6e":0}}
86382025-12-16T03:04:09.596ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
86392025-12-16T03:04:09.596ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86402025-12-16T03:04:09.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86412025-12-16T03:04:09.598ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 385.340502ms iteration = 18 status = {"physical_disks_added":0}
86422025-12-16T03:04:09.599ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86432025-12-16T03:04:09.600ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 429.45151ms iteration = 11 status = {}
86442025-12-16T03:04:09.603ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
86452025-12-16T03:04:09.603ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86462025-12-16T03:04:09.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86472025-12-16T03:04:09.606ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86482025-12-16T03:04:09.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86492025-12-16T03:04:09.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86502025-12-16T03:04:09.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86512025-12-16T03:04:09.609ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
86522025-12-16T03:04:09.609ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
86532025-12-16T03:04:09.609ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7d2f197-a30f-482e-904a-0ed9f1a9a822", "content-length": "372", "date": "Tue, 16 Dec 2025 03:04:09 GMT"} })
86542025-12-16T03:04:09.609ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
86552025-12-16T03:04:09.610ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
86562025-12-16T03:04:09.610ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
86572025-12-16T03:04:09.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
86582025-12-16T03:04:09.610ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86592025-12-16T03:04:09.610ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "679da276-e278-439e-aa5e-80cdf6c93ca5", "content-length": "26", "date": "Tue, 16 Dec 2025 03:04:09 GMT"} })
86602025-12-16T03:04:09.610ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
86612025-12-16T03:04:09.613ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
86622025-12-16T03:04:09.619ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
86632025-12-16T03:04:09.633ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
86642025-12-16T03:04:09.633ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
86652025-12-16T03:04:09.633ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "86d9d948-0bb7-4429-ae38-cebcd86c76a2", "date": "Tue, 16 Dec 2025 03:04:09 GMT"} })
86662025-12-16T03:04:09.633ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
86672025-12-16T03:04:09.634ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
86682025-12-16T03:04:09.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
86692025-12-16T03:04:09.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
86702025-12-16T03:04:09.636ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86712025-12-16T03:04:09.637ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
86722025-12-16T03:04:09.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
86732025-12-16T03:04:09.641ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
86742025-12-16T03:04:09.641ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
86752025-12-16T03:04:09.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
86762025-12-16T03:04:09.642ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
86772025-12-16T03:04:09.643ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
86782025-12-16T03:04:09.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
86792025-12-16T03:04:09.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
86802025-12-16T03:04:09.643ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86812025-12-16T03:04:09.645ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86822025-12-16T03:04:09.647ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
86832025-12-16T03:04:09.647ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
86842025-12-16T03:04:09.648ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "fb22fa12-96df-49ed-b655-e377f456c6d8", "date": "Tue, 16 Dec 2025 03:04:09 GMT"} })
86852025-12-16T03:04:09.649ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
86862025-12-16T03:04:09.649ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
86872025-12-16T03:04:09.649ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
86882025-12-16T03:04:09.649ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
86892025-12-16T03:04:09.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
86902025-12-16T03:04:09.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
86912025-12-16T03:04:09.650ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 436.907116ms iteration = 16 status = {}
86922025-12-16T03:04:09.650ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 17 reason = Dependency
86932025-12-16T03:04:09.650ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
86942025-12-16T03:04:09.650ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
86952025-12-16T03:04:09.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
86962025-12-16T03:04:09.650ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b3d05c1-ce7f-403d-9280-dcc4f31baae8 step component = SledAgent step id = Ensure
86972025-12-16T03:04:09.651ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
86982025-12-16T03:04:09.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
86992025-12-16T03:04:09.652ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87002025-12-16T03:04:09.653ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
87012025-12-16T03:04:09.653ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87022025-12-16T03:04:09.657ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
87032025-12-16T03:04:09.657ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
87042025-12-16T03:04:09.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
87052025-12-16T03:04:09.657ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
87062025-12-16T03:04:09.657ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
87072025-12-16T03:04:09.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
87082025-12-16T03:04:09.658ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87092025-12-16T03:04:09.660ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
87102025-12-16T03:04:09.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
87112025-12-16T03:04:09.663ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
87122025-12-16T03:04:09.663ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
87132025-12-16T03:04:09.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
87142025-12-16T03:04:09.663ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87152025-12-16T03:04:09.666ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
87162025-12-16T03:04:09.666ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
87172025-12-16T03:04:09.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
87182025-12-16T03:04:09.668ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/artifacts-config
87192025-12-16T03:04:09.668ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts-config", status: 204, headers: {"x-request-id": "5afdaa33-f9ed-4f3f-ab92-a443739a1edc", "date": "Tue, 16 Dec 2025 03:04:09 GMT"} })
87202025-12-16T03:04:09.668ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
87212025-12-16T03:04:09.668ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4f1dda4-94b9-4e6d-ba69-41c88da8ac58", "content-length": "26", "date": "Tue, 16 Dec 2025 03:04:09 GMT"} })
87222025-12-16T03:04:09.669ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:679 sled = http://[fd00:1122:3344:101::1]:12345
87232025-12-16T03:04:09.674ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 444.554702ms iteration = 2 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"b1c76b61-acee-4be6-b582-973c938d6b6e","time":"2025-12-16T03:04:09.668750966Z"},{"operation":"list","target_sled":"b1c76b61-acee-4be6-b582-973c938d6b6e","time":"2025-12-16T03:04:09.669002590Z"}]}
87242025-12-16T03:04:09.704ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87252025-12-16T03:04:09.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87262025-12-16T03:04:09.705ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87272025-12-16T03:04:09.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87282025-12-16T03:04:09.708ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87292025-12-16T03:04:09.712ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
87302025-12-16T03:04:09.712ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 498.986212ms iteration = 6 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
87312025-12-16T03:04:09.715ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
87322025-12-16T03:04:09.717ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87332025-12-16T03:04:09.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87342025-12-16T03:04:09.720ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87352025-12-16T03:04:09.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87362025-12-16T03:04:09.722ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87372025-12-16T03:04:09.726ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
87382025-12-16T03:04:09.726ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87392025-12-16T03:04:09.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87402025-12-16T03:04:09.727ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1257, tv_nsec: 341474956 } })
87412025-12-16T03:04:09.729ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87422025-12-16T03:04:09.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87432025-12-16T03:04:09.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87442025-12-16T03:04:09.735ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
87452025-12-16T03:04:09.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87462025-12-16T03:04:09.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87472025-12-16T03:04:09.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87482025-12-16T03:04:09.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87492025-12-16T03:04:09.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87502025-12-16T03:04:09.746ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
87512025-12-16T03:04:09.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87522025-12-16T03:04:09.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87532025-12-16T03:04:09.749ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87542025-12-16T03:04:09.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87552025-12-16T03:04:09.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87562025-12-16T03:04:09.756ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
87572025-12-16T03:04:09.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87582025-12-16T03:04:09.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87592025-12-16T03:04:09.804ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
87602025-12-16T03:04:09.805ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "69eb1914-3580-4cf1-b5d0-e8f17142f714", "content-length": "691", "date": "Tue, 16 Dec 2025 03:04:09 GMT"} })
87612025-12-16T03:04:09.805ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
87622025-12-16T03:04:09.805ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
87632025-12-16T03:04:09.805ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
87642025-12-16T03:04:09.808ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 594.618803ms iteration = 11 status = {}
87652025-12-16T03:04:09.814ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), time_created: 2025-12-16T02:59:15.195809Z, time_modified: 2025-12-16T02:59:15.195809Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
87662025-12-16T03:04:09.814ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
87672025-12-16T03:04:09.815ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
87682025-12-16T03:04:09.815ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
87692025-12-16T03:04:09.815ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 0789c47d-e947-4c08-ac6a-140493083d11 parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
87702025-12-16T03:04:09.815ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87712025-12-16T03:04:09.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87722025-12-16T03:04:09.817ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87732025-12-16T03:04:09.821ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
87742025-12-16T03:04:09.821ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87752025-12-16T03:04:09.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87762025-12-16T03:04:09.823ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
87772025-12-16T03:04:09.824ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
87782025-12-16T03:04:09.824ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
87792025-12-16T03:04:09.824ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 611.6874ms iteration = 15 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
87802025-12-16T03:04:09.825ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "de3a9472-de3b-40f0-9a96-a4ae96fb3486", "date": "Tue, 16 Dec 2025 03:04:09 GMT"} }) sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
87812025-12-16T03:04:09.829ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b3d05c1-ce7f-403d-9280-dcc4f31baae8 step component = FirewallRules step id = Ensure
87822025-12-16T03:04:09.829ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87832025-12-16T03:04:09.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87842025-12-16T03:04:09.831ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87852025-12-16T03:04:09.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87862025-12-16T03:04:09.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87872025-12-16T03:04:09.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87882025-12-16T03:04:09.837ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87892025-12-16T03:04:09.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87902025-12-16T03:04:09.839ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87912025-12-16T03:04:09.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87922025-12-16T03:04:09.841ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87932025-12-16T03:04:09.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87942025-12-16T03:04:09.843ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87952025-12-16T03:04:09.847ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
87962025-12-16T03:04:09.847ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87972025-12-16T03:04:09.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87982025-12-16T03:04:09.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87992025-12-16T03:04:09.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88002025-12-16T03:04:09.851ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88012025-12-16T03:04:09.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88022025-12-16T03:04:09.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88032025-12-16T03:04:09.859ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
88042025-12-16T03:04:09.859ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88052025-12-16T03:04:09.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88062025-12-16T03:04:09.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88072025-12-16T03:04:09.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88082025-12-16T03:04:09.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88092025-12-16T03:04:09.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88102025-12-16T03:04:09.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88112025-12-16T03:04:09.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88122025-12-16T03:04:09.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88132025-12-16T03:04:09.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88142025-12-16T03:04:09.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88152025-12-16T03:04:09.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88162025-12-16T03:04:09.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88172025-12-16T03:04:09.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88182025-12-16T03:04:09.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88192025-12-16T03:04:09.880ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
88202025-12-16T03:04:09.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88212025-12-16T03:04:09.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88222025-12-16T03:04:09.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88232025-12-16T03:04:09.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88242025-12-16T03:04:09.883ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1257, tv_nsec: 497527823 } })
88252025-12-16T03:04:09.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88262025-12-16T03:04:09.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88272025-12-16T03:04:09.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88282025-12-16T03:04:09.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88292025-12-16T03:04:09.888ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
88302025-12-16T03:04:09.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88312025-12-16T03:04:09.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88322025-12-16T03:04:09.889ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "3fb6fd72-afe4-46e1-9e20-d90ab2bb336a", "date": "Tue, 16 Dec 2025 03:04:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
88332025-12-16T03:04:09.889ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
88342025-12-16T03:04:09.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88352025-12-16T03:04:09.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88362025-12-16T03:04:09.891ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "be08d9b5-94f0-46c8-94cd-ddbd493a5d20", "date": "Tue, 16 Dec 2025 03:04:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
88372025-12-16T03:04:09.891ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
88382025-12-16T03:04:09.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88392025-12-16T03:04:09.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88402025-12-16T03:04:09.893ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "75bf9300-392c-4c85-8c4d-4285d4098b74", "date": "Tue, 16 Dec 2025 03:04:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
88412025-12-16T03:04:09.893ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
88422025-12-16T03:04:09.896ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
88432025-12-16T03:04:09.896ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 724.837144ms iteration = 8 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
88442025-12-16T03:04:09.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88452025-12-16T03:04:09.904ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
88462025-12-16T03:04:09.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88472025-12-16T03:04:09.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88482025-12-16T03:04:09.904ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
88492025-12-16T03:04:09.904ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
88502025-12-16T03:04:09.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88512025-12-16T03:04:09.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88522025-12-16T03:04:09.907ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88532025-12-16T03:04:09.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88542025-12-16T03:04:09.908ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "98e17ede-1710-4ac5-8241-76f97cec1e07", "content-length": "372", "date": "Tue, 16 Dec 2025 03:04:09 GMT"} })
88552025-12-16T03:04:09.908ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
88562025-12-16T03:04:09.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
88572025-12-16T03:04:09.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88582025-12-16T03:04:09.915ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
88592025-12-16T03:04:09.915ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88602025-12-16T03:04:09.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88612025-12-16T03:04:09.919ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88622025-12-16T03:04:09.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88632025-12-16T03:04:09.921ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88642025-12-16T03:04:09.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88652025-12-16T03:04:09.921ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
88662025-12-16T03:04:09.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
88672025-12-16T03:04:09.923ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88682025-12-16T03:04:09.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88692025-12-16T03:04:09.926ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88702025-12-16T03:04:09.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88712025-12-16T03:04:09.926ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
88722025-12-16T03:04:09.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
88732025-12-16T03:04:09.927ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88742025-12-16T03:04:09.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88752025-12-16T03:04:09.928ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88762025-12-16T03:04:09.929ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88772025-12-16T03:04:09.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88782025-12-16T03:04:09.930ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88792025-12-16T03:04:09.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88802025-12-16T03:04:09.931ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
88812025-12-16T03:04:09.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88822025-12-16T03:04:09.934ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
88832025-12-16T03:04:09.937ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
88842025-12-16T03:04:09.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
88852025-12-16T03:04:09.938ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
88862025-12-16T03:04:09.938ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88872025-12-16T03:04:09.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88882025-12-16T03:04:09.940ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
88892025-12-16T03:04:09.940ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
88902025-12-16T03:04:09.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
88912025-12-16T03:04:09.941ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88922025-12-16T03:04:09.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88932025-12-16T03:04:09.942ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "fa78a523-3ac6-480c-837d-2917fd2edb8b", "date": "Tue, 16 Dec 2025 03:04:09 GMT"} })
88942025-12-16T03:04:09.943ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88952025-12-16T03:04:09.944ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:506
88962025-12-16T03:04:09.944ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 730.086731ms
88972025-12-16T03:04:09.944ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 730.625097ms iteration = 2 status = {}
88982025-12-16T03:04:09.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88992025-12-16T03:04:09.945ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89002025-12-16T03:04:09.951ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
89012025-12-16T03:04:09.951ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89022025-12-16T03:04:09.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89032025-12-16T03:04:09.955ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
89042025-12-16T03:04:09.955ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
89052025-12-16T03:04:09.956ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
89062025-12-16T03:04:09.956ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
89072025-12-16T03:04:09.956ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
89082025-12-16T03:04:09.956ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
89092025-12-16T03:04:09.956ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
89102025-12-16T03:04:09.956ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
89112025-12-16T03:04:09.957ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
89122025-12-16T03:04:09.957ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
89132025-12-16T03:04:09.957ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
89142025-12-16T03:04:09.957ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89152025-12-16T03:04:09.957ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }], Name("external-dns"): [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
89162025-12-16T03:04:09.957ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89172025-12-16T03:04:09.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89182025-12-16T03:04:09.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89192025-12-16T03:04:09.959ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89202025-12-16T03:04:09.960ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
89212025-12-16T03:04:09.960ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
89222025-12-16T03:04:09.962ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
89232025-12-16T03:04:09.962ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89242025-12-16T03:04:09.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89252025-12-16T03:04:09.963ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
89262025-12-16T03:04:09.963ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
89272025-12-16T03:04:09.963ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
89282025-12-16T03:04:09.973ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
89292025-12-16T03:04:09.973ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
89302025-12-16T03:04:09.974ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "7bee1e88-3bd1-4688-a2cd-81ef529d6ebc", "date": "Tue, 16 Dec 2025 03:04:09 GMT"} })
89312025-12-16T03:04:09.974ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
89322025-12-16T03:04:09.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
89332025-12-16T03:04:09.977ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
89342025-12-16T03:04:09.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
89352025-12-16T03:04:09.982ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
89362025-12-16T03:04:09.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
89372025-12-16T03:04:09.985ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
89382025-12-16T03:04:09.985ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
89392025-12-16T03:04:09.990ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "2a300b0c-4428-4308-bb7b-8c4c939a4f53", "date": "Tue, 16 Dec 2025 03:04:09 GMT"} })
89402025-12-16T03:04:09.992ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 342.52334ms iteration = 17 status = {}
89412025-12-16T03:04:10.118ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), time_created: 2025-12-16T02:59:15.195809Z, time_modified: 2025-12-16T02:59:15.195809Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
89422025-12-16T03:04:10.118ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
89432025-12-16T03:04:10.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89442025-12-16T03:04:10.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89452025-12-16T03:04:10.155ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89462025-12-16T03:04:10.155ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89472025-12-16T03:04:10.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89482025-12-16T03:04:10.160ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
89492025-12-16T03:04:10.160ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
89502025-12-16T03:04:10.166ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "a3d0eca0-0411-4b2b-b7fd-6c2839898893", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
89512025-12-16T03:04:10.169ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
89522025-12-16T03:04:10.169ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b3d05c1-ce7f-403d-9280-dcc4f31baae8 step component = Dns step id = Ensure
89532025-12-16T03:04:10.169ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
89542025-12-16T03:04:10.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89552025-12-16T03:04:10.169ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89562025-12-16T03:04:10.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89572025-12-16T03:04:10.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89582025-12-16T03:04:10.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89592025-12-16T03:04:10.175ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
89602025-12-16T03:04:10.217ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
89612025-12-16T03:04:10.217ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
89622025-12-16T03:04:10.217ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
89632025-12-16T03:04:10.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89642025-12-16T03:04:10.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89652025-12-16T03:04:10.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89662025-12-16T03:04:10.222ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
89672025-12-16T03:04:10.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89682025-12-16T03:04:10.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89692025-12-16T03:04:10.225ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
89702025-12-16T03:04:10.225ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
89712025-12-16T03:04:10.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89722025-12-16T03:04:10.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89732025-12-16T03:04:10.227ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89742025-12-16T03:04:10.230ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
89752025-12-16T03:04:10.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89762025-12-16T03:04:10.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89772025-12-16T03:04:10.232ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
89782025-12-16T03:04:10.232ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
89792025-12-16T03:04:10.232ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
89802025-12-16T03:04:10.232ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89812025-12-16T03:04:10.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89822025-12-16T03:04:10.233ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89832025-12-16T03:04:10.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89842025-12-16T03:04:10.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89852025-12-16T03:04:10.239ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
89862025-12-16T03:04:10.249ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "adcf6597-9269-434a-9fd3-278de029f7d7", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
89872025-12-16T03:04:10.249ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
89882025-12-16T03:04:10.249ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
89892025-12-16T03:04:10.250ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "4cb22e2a-2429-4cc9-896a-d9a036d5d91e", "content-length": "682", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
89902025-12-16T03:04:10.250ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89912025-12-16T03:04:10.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89922025-12-16T03:04:10.254ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89932025-12-16T03:04:10.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89942025-12-16T03:04:10.258ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89952025-12-16T03:04:10.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89962025-12-16T03:04:10.260ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89972025-12-16T03:04:10.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89982025-12-16T03:04:10.268ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.054717654s iteration = 11 status = {}
89992025-12-16T03:04:10.280ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
90002025-12-16T03:04:10.280ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
90012025-12-16T03:04:10.280ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
90022025-12-16T03:04:10.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90032025-12-16T03:04:10.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90042025-12-16T03:04:10.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90052025-12-16T03:04:10.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
90062025-12-16T03:04:10.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90072025-12-16T03:04:10.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90082025-12-16T03:04:10.287ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
90092025-12-16T03:04:10.287ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
90102025-12-16T03:04:10.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90112025-12-16T03:04:10.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90122025-12-16T03:04:10.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90132025-12-16T03:04:10.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
90142025-12-16T03:04:10.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90152025-12-16T03:04:10.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90162025-12-16T03:04:10.294ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
90172025-12-16T03:04:10.294ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
90182025-12-16T03:04:10.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90192025-12-16T03:04:10.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90202025-12-16T03:04:10.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90212025-12-16T03:04:10.298ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
90222025-12-16T03:04:10.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90232025-12-16T03:04:10.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90242025-12-16T03:04:10.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90252025-12-16T03:04:10.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90262025-12-16T03:04:10.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90272025-12-16T03:04:10.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90282025-12-16T03:04:10.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90292025-12-16T03:04:10.306ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
90302025-12-16T03:04:10.307ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
90312025-12-16T03:04:10.307ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
90322025-12-16T03:04:10.307ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b3d05c1-ce7f-403d-9280-dcc4f31baae8 step component = OmicronZones step id = Cleanup
90332025-12-16T03:04:10.307ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b3d05c1-ce7f-403d-9280-dcc4f31baae8 step component = OmicronZones step id = Cleanup
90342025-12-16T03:04:10.307ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b3d05c1-ce7f-403d-9280-dcc4f31baae8 step component = PhysicalDisks step id = Cleanup
90352025-12-16T03:04:10.307ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b3d05c1-ce7f-403d-9280-dcc4f31baae8 step component = Clickhouse step id = Ensure
90362025-12-16T03:04:10.307ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b3d05c1-ce7f-403d-9280-dcc4f31baae8 step component = Clickhouse step id = Ensure
90372025-12-16T03:04:10.338ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
90382025-12-16T03:04:10.343ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "efd40fef-0479-4b40-a1d8-78cc08a57cfb", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
90392025-12-16T03:04:10.346ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b3d05c1-ce7f-403d-9280-dcc4f31baae8 step component = SupportBundles step id = Cleanup
90402025-12-16T03:04:10.346ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90412025-12-16T03:04:10.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90422025-12-16T03:04:10.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90432025-12-16T03:04:10.349ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
90442025-12-16T03:04:10.349ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90452025-12-16T03:04:10.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90462025-12-16T03:04:10.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90472025-12-16T03:04:10.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90482025-12-16T03:04:10.352ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90492025-12-16T03:04:10.355ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
90502025-12-16T03:04:10.403ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b3d05c1-ce7f-403d-9280-dcc4f31baae8 step component = OmicronZones step id = Cleanup
90512025-12-16T03:04:10.403ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
90522025-12-16T03:04:10.403ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
90532025-12-16T03:04:10.403ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90542025-12-16T03:04:10.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90552025-12-16T03:04:10.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90562025-12-16T03:04:10.407ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
90572025-12-16T03:04:10.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90582025-12-16T03:04:10.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90592025-12-16T03:04:10.457ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
90602025-12-16T03:04:10.457ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(a91734db-9ac7-4793-9c32-ddbb21f8d986 (blueprint), false)
90612025-12-16T03:04:10.457ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b3d05c1-ce7f-403d-9280-dcc4f31baae8 step component = Cockroach step id = Ensure
90622025-12-16T03:04:10.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90632025-12-16T03:04:10.458ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90642025-12-16T03:04:10.478ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
90652025-12-16T03:04:10.478ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6b3d05c1-ce7f-403d-9280-dcc4f31baae8 step component = MgsUpdates step id = Ensure
90662025-12-16T03:04:10.478ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 17 reason = Dependency
90672025-12-16T03:04:10.478ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 7 reason = Dependency
90682025-12-16T03:04:10.478ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
90692025-12-16T03:04:10.478ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90702025-12-16T03:04:10.478ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.307137582s iteration = 7 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"e61559e9-b543-4f97-bb9f-770804745d83","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"e61559e9-b543-4f97-bb9f-770804745d83","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":4310,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":116958522,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":116958482,"secs":0}},"event_index":1,"execution_id":"e61559e9-b543-4f97-bb9f-770804745d83","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":116976252,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":264558091,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":264558051,"secs":0}},"event_index":2,"execution_id":"e61559e9-b543-4f97-bb9f-770804745d83","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":381556554,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":97724338,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":97724298,"secs":0}},"event_index":3,"execution_id":"e61559e9-b543-4f97-bb9f-770804745d83","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":479298533,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":178452301,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":178452261,"secs":0}},"event_index":4,"execution_id":"e61559e9-b543-4f97-bb9f-770804745d83","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":657784354,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":339924547,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":339924507,"secs":0}},"event_index":5,"execution_id":"e61559e9-b543-4f97-bb9f-770804745d83","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":997757232,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":138537695,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":138537666,"secs":0}},"event_index":6,"execution_id":"e61559e9-b543-4f97-bb9f-770804745d83","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":136308518,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4260,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4230,"secs":0}},"event_index":7,"execution_id":"e61559e9-b543-4f97-bb9f-770804745d83","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":136324448,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2460,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2430,"secs":0}},"event_index":8,"execution_id":"e61559e9-b543-4f97-bb9f-770804745d83","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":136329458,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3200,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3160,"secs":0}},"event_index":9,"execution_id":"e61559e9-b543-4f97-bb9f-770804745d83","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":136334558,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2430,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":2380,"secs":0}},"event_index":10,"execution_id":"e61559e9-b543-4f97-bb9f-770804745d83","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":136338648,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":38320015,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":38319986,"secs":0}},"event_index":11,"execution_id":"e61559e9-b543-4f97-bb9f-770804745d83","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":174664393,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":57386197,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":57386157,"secs":0}},"event_index":12,"execution_id":"e61559e9-b543-4f97-bb9f-770804745d83","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":232064971,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":54352508,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":54352468,"secs":0}},"event_index":13,"execution_id":"e61559e9-b543-4f97-bb9f-770804745d83","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":286427739,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":20166236,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":20166196,"secs":0}},"event_index":14,"execution_id":"e61559e9-b543-4f97-bb9f-770804745d83","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":306608575,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5030,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5000,"secs":0}},"event_index":15,"execution_id":"e61559e9-b543-4f97-bb9f-770804745d83","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":306622835,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986"}
90712025-12-16T03:04:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
90722025-12-16T03:04:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
90732025-12-16T03:04:10.480ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90742025-12-16T03:04:10.480ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90752025-12-16T03:04:10.483ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
90762025-12-16T03:04:10.483ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90772025-12-16T03:04:10.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
90782025-12-16T03:04:10.483ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
90792025-12-16T03:04:10.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
90802025-12-16T03:04:10.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
90812025-12-16T03:04:10.524ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
90822025-12-16T03:04:10.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
90832025-12-16T03:04:10.526ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90842025-12-16T03:04:10.528ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
90852025-12-16T03:04:10.528ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
90862025-12-16T03:04:10.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
90872025-12-16T03:04:10.531ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
90882025-12-16T03:04:10.531ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 52.937281ms iteration = 17 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
90892025-12-16T03:04:10.608ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(1abd4e89-7a86-47f9-96d1-3b1926a7b67c, 0), (58e21b5f-57d6-4b9f-861f-24cabd63d443, 0), (b48a1e8c-4c75-4571-b0b8-834af8e60fa2, 0)]
90902025-12-16T03:04:10.609ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1258, tv_nsec: 222700521 } })
90912025-12-16T03:04:10.645ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("20488683-5778-4090-b36e-e6f0d80a682b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("35dbaad1-8536-44a0-8a0a-35d7c39dcfbb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b57fa226-2428-4a54-b85d-60409e28b484.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("d812a3e2-48ae-4812-a1cc-7eb7623e4e00.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::5)) }], valid_until: Instant { tv_sec: 1258, tv_nsec: 258948961 } })
90922025-12-16T03:04:10.646ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
90932025-12-16T03:04:10.646ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
90942025-12-16T03:04:10.646ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
90952025-12-16T03:04:10.646ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "d33eecc4-3461-46da-820b-61015eb737fb", "content-length": "476", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
90962025-12-16T03:04:10.646ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
90972025-12-16T03:04:10.646ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d23645d9-f181-472c-bb79-5244fc7d1334", "content-length": "734", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
90982025-12-16T03:04:10.646ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
90992025-12-16T03:04:10.647ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1585b9cc-e0ac-42f2-894f-a6d64bc1c9ac", "content-length": "111", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91002025-12-16T03:04:10.647ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
91012025-12-16T03:04:10.647ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e147aca-c689-4c87-a3fb-4082d4c5c89d", "content-length": "111", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91022025-12-16T03:04:10.647ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
91032025-12-16T03:04:10.648ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9e00e2b-1105-4e56-905c-868b1d59800a", "content-length": "163", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91042025-12-16T03:04:10.648ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
91052025-12-16T03:04:10.648ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f061d86-9962-4553-90d9-7a9b91057ff5", "content-length": "163", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91062025-12-16T03:04:10.648ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
91072025-12-16T03:04:10.649ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "828ffc58-2019-4e69-8132-110dcb314a3b", "content-length": "166", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91082025-12-16T03:04:10.649ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
91092025-12-16T03:04:10.649ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "18761cd4-e6d8-4ff9-b78c-e9c39ead4b51", "content-length": "166", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91102025-12-16T03:04:10.649ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
91112025-12-16T03:04:10.649ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f31874c4-1cc9-46c3-a3a3-3994cdef7953", "content-length": "702", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91122025-12-16T03:04:10.650ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
91132025-12-16T03:04:10.650ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ad516a9-b39c-4207-abcf-c9a1b4287d9a", "content-length": "727", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91142025-12-16T03:04:10.650ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
91152025-12-16T03:04:10.650ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e798f16-517e-4687-9e6d-fa38d9898517", "content-length": "729", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91162025-12-16T03:04:10.650ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
91172025-12-16T03:04:10.650ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e004c15b-f989-4d4f-a55f-91295a7a25ec", "content-length": "728", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91182025-12-16T03:04:10.650ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
91192025-12-16T03:04:10.651ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8243d38-252c-428e-9dd0-fdcf6f11ec40", "content-length": "722", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91202025-12-16T03:04:10.651ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
91212025-12-16T03:04:10.651ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "6eca93fc-c021-41c4-92ac-673d1c574d56", "content-length": "10", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91222025-12-16T03:04:10.651ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
91232025-12-16T03:04:10.651ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8264f806-63a7-49fd-885a-538b47eeff25", "content-length": "144", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91242025-12-16T03:04:10.651ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
91252025-12-16T03:04:10.651ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d21d0fab-fb91-4eb1-95ee-6ceafc97a30b", "content-length": "147", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91262025-12-16T03:04:10.651ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
91272025-12-16T03:04:10.652ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "deaa984f-ed15-4ea5-b258-26eaa3515d20", "content-length": "109", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91282025-12-16T03:04:10.652ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
91292025-12-16T03:04:10.652ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b029eab-b00d-4f86-972c-4b7cbb35ebd8", "content-length": "109", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91302025-12-16T03:04:10.652ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
91312025-12-16T03:04:10.653ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e8b6eb0-0276-4bfb-8e53-608aa1899d1c", "content-length": "163", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91322025-12-16T03:04:10.653ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
91332025-12-16T03:04:10.653ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8bcc5b28-91e8-4fe3-a12f-2ae28329caf2", "content-length": "163", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91342025-12-16T03:04:10.653ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
91352025-12-16T03:04:10.653ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b847f36-3479-4e99-81d4-dfade7bee795", "content-length": "166", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91362025-12-16T03:04:10.653ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
91372025-12-16T03:04:10.654ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "157aafac-dfb8-48ab-aeb5-b46797413333", "content-length": "166", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91382025-12-16T03:04:10.654ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
91392025-12-16T03:04:10.654ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0be43e4d-c5cc-4555-9ba8-f5cc79cd505b", "content-length": "702", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91402025-12-16T03:04:10.654ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
91412025-12-16T03:04:10.654ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1705511c-49f9-413f-a627-6d6a4767342a", "content-length": "727", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91422025-12-16T03:04:10.654ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
91432025-12-16T03:04:10.654ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "097fa75e-6e04-4477-b3a6-14b8e896e0d1", "content-length": "729", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91442025-12-16T03:04:10.654ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
91452025-12-16T03:04:10.654ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "48d0f36e-55d7-408e-9a62-957961b37162", "content-length": "728", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91462025-12-16T03:04:10.654ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
91472025-12-16T03:04:10.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
91482025-12-16T03:04:10.657ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91492025-12-16T03:04:10.662ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
91502025-12-16T03:04:10.662ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
91512025-12-16T03:04:10.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
91522025-12-16T03:04:10.665ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91532025-12-16T03:04:10.667ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
91542025-12-16T03:04:10.667ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
91552025-12-16T03:04:10.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
91562025-12-16T03:04:10.702ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
91572025-12-16T03:04:10.702ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
91582025-12-16T03:04:10.831ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bac1bd1-64fb-4383-b699-93b19ec70aad", "content-length": "95612", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91592025-12-16T03:04:10.833ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
91602025-12-16T03:04:10.833ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
91612025-12-16T03:04:10.833ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
91622025-12-16T03:04:10.834ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
91632025-12-16T03:04:10.834ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
91642025-12-16T03:04:10.834ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
91652025-12-16T03:04:10.834ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
91662025-12-16T03:04:10.834ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
91672025-12-16T03:04:10.834ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d43bcf13-a3ff-47af-9f48-2b14e3c51af1", "content-length": "64", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91682025-12-16T03:04:10.834ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0000931-d71f-4d46-addb-bafc8abe325e", "content-length": "64", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91692025-12-16T03:04:10.834ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
91702025-12-16T03:04:10.834ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
91712025-12-16T03:04:10.835ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ac4bc16-17d0-492d-adbe-5bf17fecd3be", "content-length": "64", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91722025-12-16T03:04:10.835ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
91732025-12-16T03:04:10.838ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a1bb26d-cd80-47c2-942f-a52c5db26441", "content-length": "64", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91742025-12-16T03:04:10.838ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4790d946-db32-43d0-b398-844a882b1f69", "content-length": "64", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91752025-12-16T03:04:10.838ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
91762025-12-16T03:04:10.838ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
91772025-12-16T03:04:10.872ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4f33e4c-15dc-43e3-821a-4fef0bb5abdb", "content-length": "427187", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91782025-12-16T03:04:10.872ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b7ab4b8-e3a1-4ed8-a29a-ea0e9c1bb5d6", "content-length": "398974", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91792025-12-16T03:04:10.873ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "6887a1ad-0c5e-4498-94b8-81f0597b8fbc", "content-length": "396314", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91802025-12-16T03:04:10.875ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a08c46c-9288-45bb-a0a2-4af66e4676f2", "content-length": "396023", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91812025-12-16T03:04:10.876ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "449526fc-0532-4046-8ade-536f9c5ecd28", "content-length": "398592", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91822025-12-16T03:04:10.878ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
91832025-12-16T03:04:10.878ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
91842025-12-16T03:04:10.878ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
91852025-12-16T03:04:10.878ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
91862025-12-16T03:04:10.878ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
91872025-12-16T03:04:10.909ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c (omicron_zone)
91882025-12-16T03:04:10.909ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
91892025-12-16T03:04:10.912ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "3af1e079-7404-49dd-b682-3b3eb1250210", "content-length": "124", "date": "Tue, 16 Dec 2025 03:04:10 GMT"} })
91902025-12-16T03:04:10.915ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
91912025-12-16T03:04:11.004ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
91922025-12-16T03:04:11.004ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
91932025-12-16T03:04:11.005ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ec93466-064c-4b22-bc66-f5d14f9a5b42", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:04:11 GMT"} })
91942025-12-16T03:04:11.005ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
91952025-12-16T03:04:11.006ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
91962025-12-16T03:04:11.006ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
91972025-12-16T03:04:11.006ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0403038-67df-46ae-9a52-bcfcc6533782", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:04:11 GMT"} })
91982025-12-16T03:04:11.007ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
91992025-12-16T03:04:11.008ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
92002025-12-16T03:04:11.008ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
92012025-12-16T03:04:11.009ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce5b112d-b2d5-4e30-a94a-191bd9450463", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:04:11 GMT"} })
92022025-12-16T03:04:11.009ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
92032025-12-16T03:04:11.010ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
92042025-12-16T03:04:11.010ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
92052025-12-16T03:04:11.013ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92062025-12-16T03:04:11.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
92072025-12-16T03:04:11.015ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92082025-12-16T03:04:11.017ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
92092025-12-16T03:04:11.017ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92102025-12-16T03:04:11.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
92112025-12-16T03:04:11.063ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 288 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 75a96a21-5795-4bf6-8bba-41967e3e1260 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
92122025-12-16T03:04:11.233ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 5252caf0-f0b7-49df-83a4-ee1dc3a503fd file = nexus/db-queries/src/db/datastore/inventory.rs:1613
92132025-12-16T03:04:11.233ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 5252caf0-f0b7-49df-83a4-ee1dc3a503fd time_started = 2025-12-16 03:04:10.645996 UTC
92142025-12-16T03:04:11.233ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 20 reason = Dependency
92152025-12-16T03:04:11.233ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
92162025-12-16T03:04:11.233ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 28 reason = Dependency
92172025-12-16T03:04:11.233ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 23.24\u{b5}s iteration = 20 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
92182025-12-16T03:04:11.233ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
92192025-12-16T03:04:11.233ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 755.164208ms iteration = 7 status = {"collection_id":"5252caf0-f0b7-49df-83a4-ee1dc3a503fd","time_done":"2025-12-16 03:04:11.010895 UTC","time_started":"2025-12-16 03:04:10.645996 UTC"}
92202025-12-16T03:04:11.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
92212025-12-16T03:04:11.236ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92222025-12-16T03:04:11.240ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
92232025-12-16T03:04:11.240ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
92242025-12-16T03:04:11.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
92252025-12-16T03:04:11.242ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
92262025-12-16T03:04:11.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
92272025-12-16T03:04:11.290ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 16 reason = Dependency
92282025-12-16T03:04:11.290ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 56.977172ms iteration = 28 status = {"Loaded":{"collection_id":"5252caf0-f0b7-49df-83a4-ee1dc3a503fd","time_loaded":"2025-12-16T03:04:11.233311550Z","time_started":"2025-12-16T03:04:10.645996Z"}}
92292025-12-16T03:04:11.290ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 10 reason = Dependency
92302025-12-16T03:04:11.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92312025-12-16T03:04:11.290ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 19 reason = Dependency
92322025-12-16T03:04:11.290ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
92332025-12-16T03:04:11.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
92342025-12-16T03:04:11.290ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 21 reason = Dependency
92352025-12-16T03:04:11.290ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
92362025-12-16T03:04:11.290ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 23.62\u{b5}s iteration = 21 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
92372025-12-16T03:04:11.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92382025-12-16T03:04:11.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
92392025-12-16T03:04:11.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
92402025-12-16T03:04:11.291ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92412025-12-16T03:04:11.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92422025-12-16T03:04:11.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92432025-12-16T03:04:11.295ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
92442025-12-16T03:04:11.295ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
92452025-12-16T03:04:11.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
92462025-12-16T03:04:11.295ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
92472025-12-16T03:04:11.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
92482025-12-16T03:04:11.296ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
92492025-12-16T03:04:11.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92502025-12-16T03:04:11.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
92512025-12-16T03:04:11.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92522025-12-16T03:04:11.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92532025-12-16T03:04:11.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92542025-12-16T03:04:11.301ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
92552025-12-16T03:04:11.301ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
92562025-12-16T03:04:11.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92572025-12-16T03:04:11.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
92582025-12-16T03:04:11.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92592025-12-16T03:04:11.302ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
92602025-12-16T03:04:11.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
92612025-12-16T03:04:11.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
92622025-12-16T03:04:11.303ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92632025-12-16T03:04:11.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
92642025-12-16T03:04:11.306ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
92652025-12-16T03:04:11.306ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
92662025-12-16T03:04:11.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
92672025-12-16T03:04:11.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
92682025-12-16T03:04:11.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92692025-12-16T03:04:11.311ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
92702025-12-16T03:04:11.311ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
92712025-12-16T03:04:11.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
92722025-12-16T03:04:11.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92732025-12-16T03:04:11.314ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
92742025-12-16T03:04:11.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
92752025-12-16T03:04:11.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
92762025-12-16T03:04:11.317ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
92772025-12-16T03:04:11.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
92782025-12-16T03:04:11.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
92792025-12-16T03:04:11.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92802025-12-16T03:04:11.321ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
92812025-12-16T03:04:11.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
92822025-12-16T03:04:11.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
92832025-12-16T03:04:11.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92842025-12-16T03:04:11.326ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
92852025-12-16T03:04:11.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
92862025-12-16T03:04:11.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
92872025-12-16T03:04:11.331ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
92882025-12-16T03:04:11.331ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 41.093031ms iteration = 10 status = {"blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","inventory_collection_id":"5252caf0-f0b7-49df-83a4-ee1dc3a503fd","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
92892025-12-16T03:04:11.363ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 73.36849ms iteration = 19 status = {"physical_disks_added":0}
92902025-12-16T03:04:11.400ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92912025-12-16T03:04:11.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92922025-12-16T03:04:11.401ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92932025-12-16T03:04:11.404ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
92942025-12-16T03:04:11.404ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92952025-12-16T03:04:11.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92962025-12-16T03:04:11.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92972025-12-16T03:04:11.410ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
92982025-12-16T03:04:11.410ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92992025-12-16T03:04:11.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93002025-12-16T03:04:11.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93012025-12-16T03:04:11.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93022025-12-16T03:04:11.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93032025-12-16T03:04:11.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93042025-12-16T03:04:11.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93052025-12-16T03:04:11.454ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
93062025-12-16T03:04:11.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93072025-12-16T03:04:11.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93082025-12-16T03:04:11.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93092025-12-16T03:04:11.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93102025-12-16T03:04:11.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93112025-12-16T03:04:11.460ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
93122025-12-16T03:04:11.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93132025-12-16T03:04:11.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93142025-12-16T03:04:11.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93152025-12-16T03:04:11.464ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
93162025-12-16T03:04:11.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93172025-12-16T03:04:11.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93182025-12-16T03:04:11.466ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93192025-12-16T03:04:11.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93202025-12-16T03:04:11.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93212025-12-16T03:04:11.470ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
93222025-12-16T03:04:11.470ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93232025-12-16T03:04:11.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93242025-12-16T03:04:11.472ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93252025-12-16T03:04:11.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93262025-12-16T03:04:11.474ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93272025-12-16T03:04:11.476ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
93282025-12-16T03:04:11.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93292025-12-16T03:04:11.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93302025-12-16T03:04:11.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93312025-12-16T03:04:11.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93322025-12-16T03:04:11.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93332025-12-16T03:04:11.482ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
93342025-12-16T03:04:11.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93352025-12-16T03:04:11.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93362025-12-16T03:04:11.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93372025-12-16T03:04:11.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93382025-12-16T03:04:11.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93392025-12-16T03:04:11.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93402025-12-16T03:04:11.497ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93412025-12-16T03:04:11.500ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
93422025-12-16T03:04:11.500ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93432025-12-16T03:04:11.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93442025-12-16T03:04:11.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93452025-12-16T03:04:11.504ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
93462025-12-16T03:04:11.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93472025-12-16T03:04:11.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93482025-12-16T03:04:11.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93492025-12-16T03:04:11.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93502025-12-16T03:04:11.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93512025-12-16T03:04:11.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93522025-12-16T03:04:11.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93532025-12-16T03:04:11.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
93542025-12-16T03:04:11.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93552025-12-16T03:04:11.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93562025-12-16T03:04:11.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93572025-12-16T03:04:11.518ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
93582025-12-16T03:04:11.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93592025-12-16T03:04:11.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93602025-12-16T03:04:11.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93612025-12-16T03:04:11.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93622025-12-16T03:04:11.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93632025-12-16T03:04:11.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93642025-12-16T03:04:11.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93652025-12-16T03:04:11.525ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
93662025-12-16T03:04:11.569ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93672025-12-16T03:04:11.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93682025-12-16T03:04:11.570ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93692025-12-16T03:04:11.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93702025-12-16T03:04:11.572ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93712025-12-16T03:04:11.575ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
93722025-12-16T03:04:11.576ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93732025-12-16T03:04:11.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93742025-12-16T03:04:11.578ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93752025-12-16T03:04:11.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93762025-12-16T03:04:11.580ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93772025-12-16T03:04:11.583ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
93782025-12-16T03:04:11.583ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93792025-12-16T03:04:11.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93802025-12-16T03:04:11.584ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93812025-12-16T03:04:11.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93822025-12-16T03:04:11.586ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93832025-12-16T03:04:11.589ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
93842025-12-16T03:04:11.589ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93852025-12-16T03:04:11.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93862025-12-16T03:04:11.591ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93872025-12-16T03:04:11.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93882025-12-16T03:04:11.592ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93892025-12-16T03:04:11.595ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
93902025-12-16T03:04:11.595ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93912025-12-16T03:04:11.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93922025-12-16T03:04:11.597ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93932025-12-16T03:04:11.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93942025-12-16T03:04:11.599ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93952025-12-16T03:04:11.601ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
93962025-12-16T03:04:11.601ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93972025-12-16T03:04:11.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93982025-12-16T03:04:11.603ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
93992025-12-16T03:04:11.603ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
94002025-12-16T03:04:11.603ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 2fe04e61-48dc-4ba5-b0a5-34256d8bbdf7 parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
94012025-12-16T03:04:11.603ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94022025-12-16T03:04:11.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94032025-12-16T03:04:11.604ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94042025-12-16T03:04:11.608ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
94052025-12-16T03:04:11.608ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94062025-12-16T03:04:11.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94072025-12-16T03:04:11.610ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
94082025-12-16T03:04:11.610ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
94092025-12-16T03:04:11.610ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 320.059225ms iteration = 16 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
94102025-12-16T03:04:14.212ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 62 reason = Timeout
94112025-12-16T03:04:14.212ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
94122025-12-16T03:04:14.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
94132025-12-16T03:04:14.215ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94142025-12-16T03:04:14.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
94152025-12-16T03:04:14.218ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
94162025-12-16T03:04:14.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
94172025-12-16T03:04:14.220ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
94182025-12-16T03:04:14.220ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.812239ms iteration = 62 status = {"config_updated":false}
94192025-12-16T03:04:19.212ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 32 reason = Timeout
94202025-12-16T03:04:19.212ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 63 reason = Timeout
94212025-12-16T03:04:19.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
94222025-12-16T03:04:19.212ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
94232025-12-16T03:04:19.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
94242025-12-16T03:04:19.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
94252025-12-16T03:04:19.213ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 32 reason = Timeout
94262025-12-16T03:04:19.213ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
94272025-12-16T03:04:19.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
94282025-12-16T03:04:19.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94292025-12-16T03:04:19.215ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94302025-12-16T03:04:19.215ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
94312025-12-16T03:04:19.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
94322025-12-16T03:04:19.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
94332025-12-16T03:04:19.218ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
94342025-12-16T03:04:19.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
94352025-12-16T03:04:19.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
94362025-12-16T03:04:19.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
94372025-12-16T03:04:19.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
94382025-12-16T03:04:19.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
94392025-12-16T03:04:19.220ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
94402025-12-16T03:04:19.220ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.806879ms iteration = 63 status = {"config_updated":false}
94412025-12-16T03:04:19.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
94422025-12-16T03:04:19.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94432025-12-16T03:04:19.225ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
94442025-12-16T03:04:19.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
94452025-12-16T03:04:19.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
94462025-12-16T03:04:19.242ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
94472025-12-16T03:04:19.242ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.660306ms iteration = 32 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
94482025-12-16T03:04:19.252ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 39.747834ms iteration = 32 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
94492025-12-16T03:04:21.063ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 298 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = b50403f9-6a56-44a9-9103-27dadce450b1 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
94502025-12-16T03:04:24.213ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 29 reason = Timeout
94512025-12-16T03:04:24.213ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 64 reason = Timeout
94522025-12-16T03:04:24.213ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
94532025-12-16T03:04:24.213ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
94542025-12-16T03:04:24.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
94552025-12-16T03:04:24.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
94562025-12-16T03:04:24.214ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94572025-12-16T03:04:24.214ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94582025-12-16T03:04:24.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
94592025-12-16T03:04:24.218ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
94602025-12-16T03:04:24.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
94612025-12-16T03:04:24.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
94622025-12-16T03:04:24.218ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
94632025-12-16T03:04:24.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
94642025-12-16T03:04:24.220ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 5252caf0-f0b7-49df-83a4-ee1dc3a503fd original_time_started = 2025-12-16 03:04:10.645996 UTC
94652025-12-16T03:04:24.220ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.564616ms iteration = 29 status = {"Loaded":{"collection_id":"5252caf0-f0b7-49df-83a4-ee1dc3a503fd","time_loaded":"2025-12-16T03:04:24.213054610Z","time_started":"2025-12-16T03:04:10.645996Z"}}
94662025-12-16T03:04:24.220ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
94672025-12-16T03:04:24.220ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.788439ms iteration = 64 status = {"config_updated":false}
94682025-12-16T03:04:24.230ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 22 reason = Timeout
94692025-12-16T03:04:24.230ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 22 reason = Timeout
94702025-12-16T03:04:24.230ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
94712025-12-16T03:04:24.230ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
94722025-12-16T03:04:24.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
94732025-12-16T03:04:24.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
94742025-12-16T03:04:24.230ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
94752025-12-16T03:04:24.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
94762025-12-16T03:04:24.232ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94772025-12-16T03:04:24.232ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94782025-12-16T03:04:24.234ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
94792025-12-16T03:04:24.234ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
94802025-12-16T03:04:24.234ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
94812025-12-16T03:04:24.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
94822025-12-16T03:04:24.236ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.501262ms iteration = 22 status = "NoSitrep"
94832025-12-16T03:04:24.236ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
94842025-12-16T03:04:24.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
94852025-12-16T03:04:24.237ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
94862025-12-16T03:04:24.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
94872025-12-16T03:04:24.239ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94882025-12-16T03:04:24.240ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
94892025-12-16T03:04:24.241ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
94902025-12-16T03:04:24.241ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 11.284253ms iteration = 22 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
94912025-12-16T03:04:29.213ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
94922025-12-16T03:04:29.213ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 65 reason = Timeout
94932025-12-16T03:04:29.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
94942025-12-16T03:04:29.213ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
94952025-12-16T03:04:29.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
94962025-12-16T03:04:29.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
94972025-12-16T03:04:29.213ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 33 reason = Timeout
94982025-12-16T03:04:29.213ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
94992025-12-16T03:04:29.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
95002025-12-16T03:04:29.214ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95012025-12-16T03:04:29.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95022025-12-16T03:04:29.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
95032025-12-16T03:04:29.218ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
95042025-12-16T03:04:29.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
95052025-12-16T03:04:29.219ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
95062025-12-16T03:04:29.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95072025-12-16T03:04:29.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
95082025-12-16T03:04:29.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95092025-12-16T03:04:29.220ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
95102025-12-16T03:04:29.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
95112025-12-16T03:04:29.220ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.663477ms iteration = 65 status = {"config_updated":false}
95122025-12-16T03:04:29.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95132025-12-16T03:04:29.225ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
95142025-12-16T03:04:29.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95152025-12-16T03:04:29.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
95162025-12-16T03:04:29.237ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
95172025-12-16T03:04:29.237ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.995044ms iteration = 33 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
95182025-12-16T03:04:29.250ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
95192025-12-16T03:04:29.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
95202025-12-16T03:04:29.251ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.742458ms iteration = 33 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
95212025-12-16T03:04:31.065ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 461 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = f0457924-0a5a-4b6b-8690-fc757d6d2be6 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
95222025-12-16T03:04:34.213ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 66 reason = Timeout
95232025-12-16T03:04:34.213ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
95242025-12-16T03:04:34.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
95252025-12-16T03:04:34.215ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95262025-12-16T03:04:34.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
95272025-12-16T03:04:34.218ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
95282025-12-16T03:04:34.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
95292025-12-16T03:04:34.221ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
95302025-12-16T03:04:34.221ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.952511ms iteration = 66 status = {"config_updated":false}
95312025-12-16T03:04:36.056ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:33820
95322025-12-16T03:04:36.056ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33820 req_id = 8e1e5de6-88d8-487d-bea2-88963e1ac183 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending version = 1.0.0
95332025-12-16T03:04:36.056ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33820 req_id = 8e1e5de6-88d8-487d-bea2-88963e1ac183 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
95342025-12-16T03:04:36.057ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33820 req_id = 8e1e5de6-88d8-487d-bea2-88963e1ac183 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
95352025-12-16T03:04:36.060ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 3510 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33820 req_id = 8e1e5de6-88d8-487d-bea2-88963e1ac183 response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending
95362025-12-16T03:04:36.060ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33820 req_id = 3468cc98-db7a-43e3-b8f8-5c886c51ab95 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D version = 1.0.0
95372025-12-16T03:04:36.060ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33820 req_id = 3468cc98-db7a-43e3-b8f8-5c886c51ab95 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
95382025-12-16T03:04:36.061ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33820 req_id = 3468cc98-db7a-43e3-b8f8-5c886c51ab95 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
95392025-12-16T03:04:36.063ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 2387 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:33820 req_id = 3468cc98-db7a-43e3-b8f8-5c886c51ab95 response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D
95402025-12-16T03:04:39.172ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
95412025-12-16T03:04:39.172ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 12 reason = Timeout
95422025-12-16T03:04:39.172ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
95432025-12-16T03:04:39.172ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
95442025-12-16T03:04:39.172ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
95452025-12-16T03:04:39.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
95462025-12-16T03:04:39.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
95472025-12-16T03:04:39.174ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1286, tv_nsec: 788190106 } })
95482025-12-16T03:04:39.175ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1286, tv_nsec: 788239647 } })
95492025-12-16T03:04:39.175ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.777568ms iteration = 12 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
95502025-12-16T03:04:39.176ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
95512025-12-16T03:04:39.176ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1286, tv_nsec: 789472612 } })
95522025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 34 reason = Timeout
95532025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
95542025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 12 reason = Timeout
95552025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
95562025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 12 reason = Timeout
95572025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 67 reason = Timeout
95582025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 16.84\u{b5}s iteration = 12 status = {"disabled":true,"errors":[],"sps":[]}
95592025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 12 reason = Timeout
95602025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 20 reason = Timeout
95612025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
95622025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
95632025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 12 reason = Timeout
95642025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
95652025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 12 reason = Timeout
95662025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
95672025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 23 reason = Timeout
95682025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
95692025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
95702025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 30 reason = Timeout
95712025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
95722025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
95732025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 23 reason = Timeout
95742025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
95752025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
95762025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 12 reason = Timeout
95772025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
95782025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 12 reason = Timeout
95792025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
95802025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 12 reason = Timeout
95812025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
95822025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
95832025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95842025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 12 reason = Timeout
95852025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
95862025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 18 reason = Timeout
95872025-12-16T03:04:39.237ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
95882025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 12 reason = Timeout
95892025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
95902025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
95912025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 18 reason = Timeout
95922025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
95932025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 12 reason = Timeout
95942025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
95952025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
95962025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
95972025-12-16T03:04:39.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
95982025-12-16T03:04:39.238ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
95992025-12-16T03:04:39.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
96002025-12-16T03:04:39.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
96012025-12-16T03:04:39.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
96022025-12-16T03:04:39.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
96032025-12-16T03:04:39.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
96042025-12-16T03:04:39.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
96052025-12-16T03:04:39.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
96062025-12-16T03:04:39.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
96072025-12-16T03:04:39.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
96082025-12-16T03:04:39.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
96092025-12-16T03:04:39.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
96102025-12-16T03:04:39.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
96112025-12-16T03:04:39.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
96122025-12-16T03:04:39.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
96132025-12-16T03:04:39.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
96142025-12-16T03:04:39.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
96152025-12-16T03:04:39.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
96162025-12-16T03:04:39.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
96172025-12-16T03:04:39.239ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
96182025-12-16T03:04:39.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
96192025-12-16T03:04:39.240ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96202025-12-16T03:04:39.240ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
96212025-12-16T03:04:39.240ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96222025-12-16T03:04:39.241ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96232025-12-16T03:04:39.241ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 68.726401ms iteration = 12 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
96242025-12-16T03:04:39.241ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96252025-12-16T03:04:39.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
96262025-12-16T03:04:39.241ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96272025-12-16T03:04:39.242ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
96282025-12-16T03:04:39.242ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96292025-12-16T03:04:39.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
96302025-12-16T03:04:39.242ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96312025-12-16T03:04:39.243ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
96322025-12-16T03:04:39.243ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
96332025-12-16T03:04:39.244ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3681c72d-9944-410c-a0c3-3e41c0c520ea", "content-length": "26", "date": "Tue, 16 Dec 2025 03:04:39 GMT"} })
96342025-12-16T03:04:39.244ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
96352025-12-16T03:04:39.245ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96362025-12-16T03:04:39.246ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
96372025-12-16T03:04:39.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
96382025-12-16T03:04:39.246ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
96392025-12-16T03:04:39.246ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
96402025-12-16T03:04:39.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
96412025-12-16T03:04:39.246ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1286, tv_nsec: 859881445 } })
96422025-12-16T03:04:39.247ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
96432025-12-16T03:04:39.247ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
96442025-12-16T03:04:39.247ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
96452025-12-16T03:04:39.247ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
96462025-12-16T03:04:39.247ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
96472025-12-16T03:04:39.247ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
96482025-12-16T03:04:39.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
96492025-12-16T03:04:39.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
96502025-12-16T03:04:39.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
96512025-12-16T03:04:39.248ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
96522025-12-16T03:04:39.248ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
96532025-12-16T03:04:39.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
96542025-12-16T03:04:39.249ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
96552025-12-16T03:04:39.249ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
96562025-12-16T03:04:39.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
96572025-12-16T03:04:39.251ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
96582025-12-16T03:04:39.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
96592025-12-16T03:04:39.252ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
96602025-12-16T03:04:39.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
96612025-12-16T03:04:39.253ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 5252caf0-f0b7-49df-83a4-ee1dc3a503fd original_time_started = 2025-12-16 03:04:10.645996 UTC
96622025-12-16T03:04:39.253ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 15.84665ms iteration = 30 status = {"Loaded":{"collection_id":"5252caf0-f0b7-49df-83a4-ee1dc3a503fd","time_loaded":"2025-12-16T03:04:39.237492417Z","time_started":"2025-12-16T03:04:10.645996Z"}}
96632025-12-16T03:04:39.253ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
96642025-12-16T03:04:39.253ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
96652025-12-16T03:04:39.253ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
96662025-12-16T03:04:39.253ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
96672025-12-16T03:04:39.253ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 16.165595ms iteration = 67 status = {"config_updated":false}
96682025-12-16T03:04:39.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
96692025-12-16T03:04:39.253ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1286, tv_nsec: 867120736 } })
96702025-12-16T03:04:39.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
96712025-12-16T03:04:39.254ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
96722025-12-16T03:04:39.254ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1286, tv_nsec: 867888616 } })
96732025-12-16T03:04:39.255ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
96742025-12-16T03:04:39.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
96752025-12-16T03:04:39.256ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 19.141513ms iteration = 12 status = 0
96762025-12-16T03:04:39.257ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
96772025-12-16T03:04:39.257ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 19.480807ms iteration = 12 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
96782025-12-16T03:04:39.257ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96792025-12-16T03:04:39.257ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 19.969703ms iteration = 23 status = "NoSitrep"
96802025-12-16T03:04:39.259ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
96812025-12-16T03:04:39.260ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
96822025-12-16T03:04:39.260ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 23.259245ms iteration = 23 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
96832025-12-16T03:04:39.307ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96842025-12-16T03:04:39.307ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 69.927856ms iteration = 12 status = {"errors":[],"finish_invoked_ok":[]}
96852025-12-16T03:04:39.307ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96862025-12-16T03:04:39.311ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
96872025-12-16T03:04:39.311ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
96882025-12-16T03:04:39.312ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 74.786788ms iteration = 34 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
96892025-12-16T03:04:39.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
96902025-12-16T03:04:39.312ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
96912025-12-16T03:04:39.312ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
96922025-12-16T03:04:39.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
96932025-12-16T03:04:39.313ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
96942025-12-16T03:04:39.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
96952025-12-16T03:04:39.315ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96962025-12-16T03:04:39.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96972025-12-16T03:04:39.315ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
96982025-12-16T03:04:39.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
96992025-12-16T03:04:39.316ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97002025-12-16T03:04:39.318ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97012025-12-16T03:04:39.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
97022025-12-16T03:04:39.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
97032025-12-16T03:04:39.319ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
97042025-12-16T03:04:39.319ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
97052025-12-16T03:04:39.319ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
97062025-12-16T03:04:39.319ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
97072025-12-16T03:04:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
97082025-12-16T03:04:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
97092025-12-16T03:04:39.320ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
97102025-12-16T03:04:39.320ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
97112025-12-16T03:04:39.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
97122025-12-16T03:04:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
97132025-12-16T03:04:39.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
97142025-12-16T03:04:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
97152025-12-16T03:04:39.320ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 83.104933ms iteration = 12 status = {"errors":[],"garbage_collect_requested":[]}
97162025-12-16T03:04:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
97172025-12-16T03:04:39.321ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
97182025-12-16T03:04:39.321ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
97192025-12-16T03:04:39.321ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
97202025-12-16T03:04:39.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
97212025-12-16T03:04:39.321ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6b7018b-e11b-410a-9330-38d50623e198", "content-length": "26", "date": "Tue, 16 Dec 2025 03:04:39 GMT"} })
97222025-12-16T03:04:39.321ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
97232025-12-16T03:04:39.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
97242025-12-16T03:04:39.322ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
97252025-12-16T03:04:39.322ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
97262025-12-16T03:04:39.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
97272025-12-16T03:04:39.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
97282025-12-16T03:04:39.323ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
97292025-12-16T03:04:39.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
97302025-12-16T03:04:39.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
97312025-12-16T03:04:39.324ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
97322025-12-16T03:04:39.324ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 86.632478ms iteration = 18 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
97332025-12-16T03:04:39.324ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 87.065573ms iteration = 12 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
97342025-12-16T03:04:39.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97352025-12-16T03:04:39.324ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 87.324246ms iteration = 12 status = {"errors":[],"requests_created_ok":[]}
97362025-12-16T03:04:39.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
97372025-12-16T03:04:39.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
97382025-12-16T03:04:39.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
97392025-12-16T03:04:39.339ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 167.681125ms iteration = 12 status = {}
97402025-12-16T03:04:39.353ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
97412025-12-16T03:04:39.353ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 116.267753ms iteration = 34 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
97422025-12-16T03:04:39.364ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 126.915859ms iteration = 12 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
97432025-12-16T03:04:39.366ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
97442025-12-16T03:04:39.366ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
97452025-12-16T03:04:39.366ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
97462025-12-16T03:04:39.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
97472025-12-16T03:04:39.371ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1286, tv_nsec: 984214570 } })
97482025-12-16T03:04:39.373ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 136.5135ms iteration = 12 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
97492025-12-16T03:04:39.374ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
97502025-12-16T03:04:39.374ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
97512025-12-16T03:04:39.375ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8577aea-781d-4c09-99d2-1b3bcffc8df6", "content-length": "372", "date": "Tue, 16 Dec 2025 03:04:39 GMT"} })
97522025-12-16T03:04:39.375ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
97532025-12-16T03:04:39.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
97542025-12-16T03:04:39.378ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 141.506593ms iteration = 20 status = {"physical_disks_added":0}
97552025-12-16T03:04:39.405ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1287, tv_nsec: 19097812 } })
97562025-12-16T03:04:39.408ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
97572025-12-16T03:04:39.408ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
97582025-12-16T03:04:39.408ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 170.825055ms iteration = 12 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
97592025-12-16T03:04:39.425ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
97602025-12-16T03:04:39.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
97612025-12-16T03:04:39.427ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97622025-12-16T03:04:39.430ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
97632025-12-16T03:04:39.430ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
97642025-12-16T03:04:39.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
97652025-12-16T03:04:39.433ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97662025-12-16T03:04:39.435ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
97672025-12-16T03:04:39.435ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
97682025-12-16T03:04:39.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
97692025-12-16T03:04:39.437ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
97702025-12-16T03:04:39.437ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
97712025-12-16T03:04:39.437ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "8f3dc73e-b4be-479d-ad59-57015d4bac16", "date": "Tue, 16 Dec 2025 03:04:39 GMT"} })
97722025-12-16T03:04:39.437ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
97732025-12-16T03:04:39.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
97742025-12-16T03:04:39.438ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
97752025-12-16T03:04:39.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
97762025-12-16T03:04:39.440ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
97772025-12-16T03:04:39.440ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
97782025-12-16T03:04:39.440ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
97792025-12-16T03:04:39.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
97802025-12-16T03:04:39.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
97812025-12-16T03:04:39.443ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
97822025-12-16T03:04:39.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
97832025-12-16T03:04:39.446ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
97842025-12-16T03:04:39.446ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
97852025-12-16T03:04:39.450ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c9858fb6-ecfe-4d13-8db0-fa70aecbec6c", "date": "Tue, 16 Dec 2025 03:04:39 GMT"} })
97862025-12-16T03:04:39.453ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 216.145978ms iteration = 18 status = {}
97872025-12-16T03:04:39.454ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
97882025-12-16T03:04:39.454ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
97892025-12-16T03:04:39.454ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
97902025-12-16T03:04:39.454ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
97912025-12-16T03:04:39.454ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
97922025-12-16T03:04:39.454ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
97932025-12-16T03:04:39.454ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
97942025-12-16T03:04:39.454ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
97952025-12-16T03:04:39.455ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
97962025-12-16T03:04:39.455ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
97972025-12-16T03:04:39.455ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
97982025-12-16T03:04:39.455ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
97992025-12-16T03:04:39.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98002025-12-16T03:04:39.458ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
98012025-12-16T03:04:39.458ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
98022025-12-16T03:04:39.470ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
98032025-12-16T03:04:39.471ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "4efac86d-f073-4c9a-85e5-f66978454af6", "content-length": "691", "date": "Tue, 16 Dec 2025 03:04:39 GMT"} })
98042025-12-16T03:04:39.471ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
98052025-12-16T03:04:39.471ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
98062025-12-16T03:04:39.471ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
98072025-12-16T03:04:39.473ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 236.363104ms iteration = 12 status = {}
98082025-12-16T03:04:39.709ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "45cd83a6-fa78-41ea-844a-034b5e376033", "date": "Tue, 16 Dec 2025 03:04:39 GMT"} })
98092025-12-16T03:04:39.709ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
98102025-12-16T03:04:39.709ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
98112025-12-16T03:04:39.710ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7c20c45-cec4-4f54-8b56-e4137c54bb9f", "content-length": "682", "date": "Tue, 16 Dec 2025 03:04:39 GMT"} })
98122025-12-16T03:04:39.710ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
98132025-12-16T03:04:39.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98142025-12-16T03:04:39.714ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
98152025-12-16T03:04:39.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98162025-12-16T03:04:39.717ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
98172025-12-16T03:04:39.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98182025-12-16T03:04:39.719ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
98192025-12-16T03:04:39.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98202025-12-16T03:04:39.726ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 488.58851ms iteration = 12 status = {}
98212025-12-16T03:04:41.064ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 330 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 6303ca3f-38ac-43ff-aa12-500a94ef3f79 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
98222025-12-16T03:04:44.239ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 68 reason = Timeout
98232025-12-16T03:04:44.239ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
98242025-12-16T03:04:44.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
98252025-12-16T03:04:44.241ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98262025-12-16T03:04:44.244ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
98272025-12-16T03:04:44.244ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
98282025-12-16T03:04:44.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
98292025-12-16T03:04:44.246ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
98302025-12-16T03:04:44.246ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.8536ms iteration = 68 status = {"config_updated":false}
98312025-12-16T03:04:49.238ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 35 reason = Timeout
98322025-12-16T03:04:49.238ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
98332025-12-16T03:04:49.238ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
98342025-12-16T03:04:49.238ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 69 reason = Timeout
98352025-12-16T03:04:49.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98362025-12-16T03:04:49.238ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
98372025-12-16T03:04:49.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
98382025-12-16T03:04:49.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
98392025-12-16T03:04:49.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
98402025-12-16T03:04:49.240ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
98412025-12-16T03:04:49.240ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98422025-12-16T03:04:49.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98432025-12-16T03:04:49.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
98442025-12-16T03:04:49.241ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.394403ms iteration = 35 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
98452025-12-16T03:04:49.244ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
98462025-12-16T03:04:49.244ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
98472025-12-16T03:04:49.244ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
98482025-12-16T03:04:49.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98492025-12-16T03:04:49.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
98502025-12-16T03:04:49.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
98512025-12-16T03:04:49.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98522025-12-16T03:04:49.246ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
98532025-12-16T03:04:49.246ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.351926ms iteration = 69 status = {"config_updated":false}
98542025-12-16T03:04:49.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
98552025-12-16T03:04:49.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98562025-12-16T03:04:49.251ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
98572025-12-16T03:04:49.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98582025-12-16T03:04:49.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
98592025-12-16T03:04:49.262ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
98602025-12-16T03:04:49.262ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.346609ms iteration = 35 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
98612025-12-16T03:04:51.064ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 314 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 1956ddd4-cb64-4d44-b4af-d2b33685ce39 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
98622025-12-16T03:04:54.239ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 31 reason = Timeout
98632025-12-16T03:04:54.239ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 24 reason = Timeout
98642025-12-16T03:04:54.239ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
98652025-12-16T03:04:54.239ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
98662025-12-16T03:04:54.239ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 24 reason = Timeout
98672025-12-16T03:04:54.239ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 70 reason = Timeout
98682025-12-16T03:04:54.239ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
98692025-12-16T03:04:54.239ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
98702025-12-16T03:04:54.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
98712025-12-16T03:04:54.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
98722025-12-16T03:04:54.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
98732025-12-16T03:04:54.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
98742025-12-16T03:04:54.240ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
98752025-12-16T03:04:54.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
98762025-12-16T03:04:54.241ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98772025-12-16T03:04:54.241ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98782025-12-16T03:04:54.241ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98792025-12-16T03:04:54.242ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98802025-12-16T03:04:54.244ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
98812025-12-16T03:04:54.245ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
98822025-12-16T03:04:54.245ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
98832025-12-16T03:04:54.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
98842025-12-16T03:04:54.245ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
98852025-12-16T03:04:54.245ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
98862025-12-16T03:04:54.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
98872025-12-16T03:04:54.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
98882025-12-16T03:04:54.246ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
98892025-12-16T03:04:54.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
98902025-12-16T03:04:54.247ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 7.810598ms iteration = 24 status = "NoSitrep"
98912025-12-16T03:04:54.247ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
98922025-12-16T03:04:54.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
98932025-12-16T03:04:54.247ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
98942025-12-16T03:04:54.248ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 5252caf0-f0b7-49df-83a4-ee1dc3a503fd original_time_started = 2025-12-16 03:04:10.645996 UTC
98952025-12-16T03:04:54.248ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.70749ms iteration = 31 status = {"Loaded":{"collection_id":"5252caf0-f0b7-49df-83a4-ee1dc3a503fd","time_loaded":"2025-12-16T03:04:54.239334527Z","time_started":"2025-12-16T03:04:10.645996Z"}}
98962025-12-16T03:04:54.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
98972025-12-16T03:04:54.248ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
98982025-12-16T03:04:54.248ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.870192ms iteration = 70 status = {"config_updated":false}
98992025-12-16T03:04:54.249ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99002025-12-16T03:04:54.251ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
99012025-12-16T03:04:54.252ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
99022025-12-16T03:04:54.252ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 12.939014ms iteration = 24 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
99032025-12-16T03:04:59.239ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 36 reason = Timeout
99042025-12-16T03:04:59.239ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
99052025-12-16T03:04:59.239ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
99062025-12-16T03:04:59.239ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 71 reason = Timeout
99072025-12-16T03:04:59.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
99082025-12-16T03:04:59.239ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
99092025-12-16T03:04:59.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
99102025-12-16T03:04:59.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
99112025-12-16T03:04:59.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
99122025-12-16T03:04:59.241ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
99132025-12-16T03:04:59.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
99142025-12-16T03:04:59.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99152025-12-16T03:04:59.241ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99162025-12-16T03:04:59.245ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
99172025-12-16T03:04:59.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
99182025-12-16T03:04:59.245ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
99192025-12-16T03:04:59.245ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
99202025-12-16T03:04:59.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
99212025-12-16T03:04:59.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
99222025-12-16T03:04:59.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
99232025-12-16T03:04:59.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
99242025-12-16T03:04:59.246ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
99252025-12-16T03:04:59.247ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.506255ms iteration = 71 status = {"config_updated":false}
99262025-12-16T03:04:59.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99272025-12-16T03:04:59.250ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
99282025-12-16T03:04:59.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
99292025-12-16T03:04:59.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
99302025-12-16T03:04:59.267ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
99312025-12-16T03:04:59.267ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.572899ms iteration = 36 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
99322025-12-16T03:04:59.282ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 43.118866ms iteration = 36 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
99332025-12-16T03:05:01.064ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 321 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 3859387a-9165-4623-8c65-d351318a2ea8 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
99342025-12-16T03:05:04.238ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 72 reason = Timeout
99352025-12-16T03:05:04.238ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
99362025-12-16T03:05:04.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
99372025-12-16T03:05:04.241ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99382025-12-16T03:05:04.244ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
99392025-12-16T03:05:04.244ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
99402025-12-16T03:05:04.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
99412025-12-16T03:05:04.246ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
99422025-12-16T03:05:04.246ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.946011ms iteration = 72 status = {"config_updated":false}
99432025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
99442025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
99452025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
99462025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
99472025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
99482025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
99492025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
99502025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 13 reason = Timeout
99512025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
99522025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
99532025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
99542025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
99552025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
99562025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
99572025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
99582025-12-16T03:05:09.172ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
99592025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
99602025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
99612025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
99622025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 7 reason = Timeout
99632025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
99642025-12-16T03:05:09.172ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-12-16 02:55:09.172325135 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
99652025-12-16T03:05:09.172ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a3b1ba8b-606a-418f-b5f7-9d063a1bf17e step component = ExternalNetworking step id = Ensure
99662025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
99672025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99682025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
99692025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99702025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
99712025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
99722025-12-16T03:05:09.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
99732025-12-16T03:05:09.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
99742025-12-16T03:05:09.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
99752025-12-16T03:05:09.173ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1a45cd23-3d0a-43eb-adce-62c7b712baf1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1316, tv_nsec: 785736415 } })
99762025-12-16T03:05:09.173ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("49a51a59-9919-48ae-8182-4070a8430160.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6af60386-8771-4401-9d4d-e93774e519f0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 1316, tv_nsec: 785744825 } })
99772025-12-16T03:05:09.173ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
99782025-12-16T03:05:09.173ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.422128ms iteration = 7 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
99792025-12-16T03:05:09.173ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
99802025-12-16T03:05:09.173ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.56351ms iteration = 7 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
99812025-12-16T03:05:09.173ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
99822025-12-16T03:05:09.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
99832025-12-16T03:05:09.174ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
99842025-12-16T03:05:09.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
99852025-12-16T03:05:09.174ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99862025-12-16T03:05:09.175ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1316, tv_nsec: 787734521 } })
99872025-12-16T03:05:09.176ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1316, tv_nsec: 789304440 } })
99882025-12-16T03:05:09.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
99892025-12-16T03:05:09.176ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99902025-12-16T03:05:09.176ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
99912025-12-16T03:05:09.177ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99922025-12-16T03:05:09.177ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1316, tv_nsec: 790014879 } })
99932025-12-16T03:05:09.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
99942025-12-16T03:05:09.178ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
99952025-12-16T03:05:09.178ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
99962025-12-16T03:05:09.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
99972025-12-16T03:05:09.180ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-12-16 02:55:09.172325135 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
99982025-12-16T03:05:09.180ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 8.204284ms iteration = 7 status = {"expiration":"2025-12-16T02:55:09.172325135Z","pruned":[]}
99992025-12-16T03:05:09.182ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100002025-12-16T03:05:09.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100012025-12-16T03:05:09.183ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
100022025-12-16T03:05:09.183ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
100032025-12-16T03:05:09.183ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
100042025-12-16T03:05:09.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
100052025-12-16T03:05:09.184ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 11.974561ms iteration = 13 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
100062025-12-16T03:05:09.184ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
100072025-12-16T03:05:09.184ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
100082025-12-16T03:05:09.184ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
100092025-12-16T03:05:09.184ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
100102025-12-16T03:05:09.184ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
100112025-12-16T03:05:09.184ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
100122025-12-16T03:05:09.184ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
100132025-12-16T03:05:09.184ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
100142025-12-16T03:05:09.184ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100152025-12-16T03:05:09.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
100162025-12-16T03:05:09.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
100172025-12-16T03:05:09.185ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100182025-12-16T03:05:09.187ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100192025-12-16T03:05:09.188ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100202025-12-16T03:05:09.189ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
100212025-12-16T03:05:09.190ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
100222025-12-16T03:05:09.190ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
100232025-12-16T03:05:09.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
100242025-12-16T03:05:09.192ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
100252025-12-16T03:05:09.193ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
100262025-12-16T03:05:09.193ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
100272025-12-16T03:05:09.193ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
100282025-12-16T03:05:09.193ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
100292025-12-16T03:05:09.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
100302025-12-16T03:05:09.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
100312025-12-16T03:05:09.194ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
100322025-12-16T03:05:09.195ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
100332025-12-16T03:05:09.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
100342025-12-16T03:05:09.197ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
100352025-12-16T03:05:09.197ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
100362025-12-16T03:05:09.197ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
100372025-12-16T03:05:09.197ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
100382025-12-16T03:05:09.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
100392025-12-16T03:05:09.198ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
100402025-12-16T03:05:09.201ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100412025-12-16T03:05:09.201ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
100422025-12-16T03:05:09.202ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100432025-12-16T03:05:09.205ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
100442025-12-16T03:05:09.206ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
100452025-12-16T03:05:09.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
100462025-12-16T03:05:09.207ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
100472025-12-16T03:05:09.207ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
100482025-12-16T03:05:09.207ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
100492025-12-16T03:05:09.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
100502025-12-16T03:05:09.213ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 17 reason = Timeout
100512025-12-16T03:05:09.213ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 7 reason = Timeout
100522025-12-16T03:05:09.213ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
100532025-12-16T03:05:09.213ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
100542025-12-16T03:05:09.213ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 4 reason = Timeout
100552025-12-16T03:05:09.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100562025-12-16T03:05:09.213ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 7 reason = Timeout
100572025-12-16T03:05:09.213ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100582025-12-16T03:05:09.213ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
100592025-12-16T03:05:09.213ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 7 reason = Timeout
100602025-12-16T03:05:09.213ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
100612025-12-16T03:05:09.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
100622025-12-16T03:05:09.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100632025-12-16T03:05:09.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100642025-12-16T03:05:09.214ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100652025-12-16T03:05:09.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
100662025-12-16T03:05:09.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
100672025-12-16T03:05:09.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100682025-12-16T03:05:09.214ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100692025-12-16T03:05:09.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100702025-12-16T03:05:09.215ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100712025-12-16T03:05:09.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100722025-12-16T03:05:09.215ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100732025-12-16T03:05:09.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100742025-12-16T03:05:09.219ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100752025-12-16T03:05:09.221ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100762025-12-16T03:05:09.224ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
100772025-12-16T03:05:09.224ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100782025-12-16T03:05:09.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100792025-12-16T03:05:09.224ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100802025-12-16T03:05:09.225ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
100812025-12-16T03:05:09.225ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
100822025-12-16T03:05:09.225ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 11.801099ms iteration = 7 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
100832025-12-16T03:05:09.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100842025-12-16T03:05:09.226ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
100852025-12-16T03:05:09.226ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
100862025-12-16T03:05:09.226ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
100872025-12-16T03:05:09.226ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
100882025-12-16T03:05:09.226ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-12-16 02:59:18.154440 UTC dns_group = external generation = 2
100892025-12-16T03:05:09.226ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 53.813012ms iteration = 7 status = {"generation":2}
100902025-12-16T03:05:09.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
100912025-12-16T03:05:09.227ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
100922025-12-16T03:05:09.230ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
100932025-12-16T03:05:09.230ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100942025-12-16T03:05:09.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100952025-12-16T03:05:09.230ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100962025-12-16T03:05:09.230ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
100972025-12-16T03:05:09.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100982025-12-16T03:05:09.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100992025-12-16T03:05:09.230ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
101002025-12-16T03:05:09.231ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 22 reason = Timeout
101012025-12-16T03:05:09.231ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
101022025-12-16T03:05:09.231ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 20.32\u{b5}s iteration = 22 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
101032025-12-16T03:05:09.231ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 7 reason = Timeout
101042025-12-16T03:05:09.231ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
101052025-12-16T03:05:09.231ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 7 reason = Timeout
101062025-12-16T03:05:09.231ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
101072025-12-16T03:05:09.231ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 7 reason = Timeout
101082025-12-16T03:05:09.231ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
101092025-12-16T03:05:09.231ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 7 reason = Timeout
101102025-12-16T03:05:09.231ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
101112025-12-16T03:05:09.231ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
101122025-12-16T03:05:09.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
101132025-12-16T03:05:09.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
101142025-12-16T03:05:09.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
101152025-12-16T03:05:09.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
101162025-12-16T03:05:09.233ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101172025-12-16T03:05:09.233ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
101182025-12-16T03:05:09.233ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
101192025-12-16T03:05:09.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
101202025-12-16T03:05:09.234ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101212025-12-16T03:05:09.234ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
101222025-12-16T03:05:09.234ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 3.058209ms iteration = 7 status = {"by_rx":{},"error":null}
101232025-12-16T03:05:09.235ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
101242025-12-16T03:05:09.236ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.427206ms iteration = 7 status = {"errors":[],"found_port_ok":[]}
101252025-12-16T03:05:09.237ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
101262025-12-16T03:05:09.237ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
101272025-12-16T03:05:09.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
101282025-12-16T03:05:09.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
101292025-12-16T03:05:09.238ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
101302025-12-16T03:05:09.238ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
101312025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
101322025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 13 reason = Timeout
101332025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 21 reason = Timeout
101342025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 9.81\u{b5}s iteration = 13 status = {"disabled":true,"errors":[],"sps":[]}
101352025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 13 reason = Timeout
101362025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
101372025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 19 reason = Timeout
101382025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
101392025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
101402025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
101412025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 13 reason = Timeout
101422025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
101432025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 13 reason = Timeout
101442025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 13 reason = Timeout
101452025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
101462025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
101472025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
101482025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
101492025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 13 reason = Timeout
101502025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
101512025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
101522025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
101532025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 13 reason = Timeout
101542025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 13 reason = Timeout
101552025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
101562025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
101572025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 32 reason = Timeout
101582025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
101592025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
101602025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 37 reason = Timeout
101612025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
101622025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
101632025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 73 reason = Timeout
101642025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
101652025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 25 reason = Timeout
101662025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
101672025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 13 reason = Timeout
101682025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
101692025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 19 reason = Timeout
101702025-12-16T03:05:09.239ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
101712025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
101722025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
101732025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
101742025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 25 reason = Timeout
101752025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
101762025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
101772025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
101782025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
101792025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
101802025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
101812025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
101822025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
101832025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
101842025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
101852025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
101862025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
101872025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
101882025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
101892025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
101902025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
101912025-12-16T03:05:09.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
101922025-12-16T03:05:09.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
101932025-12-16T03:05:09.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
101942025-12-16T03:05:09.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
101952025-12-16T03:05:09.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
101962025-12-16T03:05:09.240ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
101972025-12-16T03:05:09.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
101982025-12-16T03:05:09.241ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
101992025-12-16T03:05:09.241ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
102002025-12-16T03:05:09.241ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
102012025-12-16T03:05:09.241ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
102022025-12-16T03:05:09.241ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
102032025-12-16T03:05:09.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
102042025-12-16T03:05:09.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
102052025-12-16T03:05:09.242ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 3.656576ms iteration = 13 status = {"errors":[],"finish_invoked_ok":[]}
102062025-12-16T03:05:09.242ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102072025-12-16T03:05:09.242ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102082025-12-16T03:05:09.243ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
102092025-12-16T03:05:09.243ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a3b1ba8b-606a-418f-b5f7-9d063a1bf17e step component = SledList step id = Fetch
102102025-12-16T03:05:09.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
102112025-12-16T03:05:09.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
102122025-12-16T03:05:09.245ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
102132025-12-16T03:05:09.245ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102142025-12-16T03:05:09.245ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102152025-12-16T03:05:09.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
102162025-12-16T03:05:09.247ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 15.317704ms iteration = 7 status = {"dispatched":[],"errors":[],"glob_version":"212.0.0","globs_reprocessed":{},"no_receivers":[]}
102172025-12-16T03:05:09.247ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
102182025-12-16T03:05:09.247ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
102192025-12-16T03:05:09.247ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
102202025-12-16T03:05:09.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
102212025-12-16T03:05:09.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
102222025-12-16T03:05:09.250ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
102232025-12-16T03:05:09.250ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
102242025-12-16T03:05:09.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
102252025-12-16T03:05:09.250ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
102262025-12-16T03:05:09.250ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
102272025-12-16T03:05:09.250ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
102282025-12-16T03:05:09.250ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
102292025-12-16T03:05:09.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
102302025-12-16T03:05:09.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
102312025-12-16T03:05:09.251ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
102322025-12-16T03:05:09.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
102332025-12-16T03:05:09.256ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102342025-12-16T03:05:09.256ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102352025-12-16T03:05:09.258ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102362025-12-16T03:05:09.261ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102372025-12-16T03:05:09.262ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
102382025-12-16T03:05:09.262ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
102392025-12-16T03:05:09.262ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102402025-12-16T03:05:09.262ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 23.370006ms iteration = 37 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
102412025-12-16T03:05:09.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
102422025-12-16T03:05:09.263ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
102432025-12-16T03:05:09.263ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
102442025-12-16T03:05:09.263ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
102452025-12-16T03:05:09.263ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
102462025-12-16T03:05:09.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
102472025-12-16T03:05:09.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
102482025-12-16T03:05:09.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102492025-12-16T03:05:09.264ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 25.060278ms iteration = 13 status = {"errors":[],"requests_created_ok":[]}
102502025-12-16T03:05:09.265ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
102512025-12-16T03:05:09.265ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
102522025-12-16T03:05:09.265ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
102532025-12-16T03:05:09.265ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102542025-12-16T03:05:09.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
102552025-12-16T03:05:09.266ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
102562025-12-16T03:05:09.266ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
102572025-12-16T03:05:09.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
102582025-12-16T03:05:09.267ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 5252caf0-f0b7-49df-83a4-ee1dc3a503fd original_time_started = 2025-12-16 03:04:10.645996 UTC
102592025-12-16T03:05:09.267ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 28.457621ms iteration = 32 status = {"Loaded":{"collection_id":"5252caf0-f0b7-49df-83a4-ee1dc3a503fd","time_loaded":"2025-12-16T03:05:09.239211042Z","time_started":"2025-12-16T03:04:10.645996Z"}}
102602025-12-16T03:05:09.267ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
102612025-12-16T03:05:09.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
102622025-12-16T03:05:09.269ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
102632025-12-16T03:05:09.269ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
102642025-12-16T03:05:09.269ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
102652025-12-16T03:05:09.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
102662025-12-16T03:05:09.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
102672025-12-16T03:05:09.269ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
102682025-12-16T03:05:09.269ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 30.418635ms iteration = 13 status = 0
102692025-12-16T03:05:09.269ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
102702025-12-16T03:05:09.269ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
102712025-12-16T03:05:09.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
102722025-12-16T03:05:09.272ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1316, tv_nsec: 885154545 } })
102732025-12-16T03:05:09.273ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
102742025-12-16T03:05:09.273ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1316, tv_nsec: 886105577 } })
102752025-12-16T03:05:09.273ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 34.441347ms iteration = 25 status = "NoSitrep"
102762025-12-16T03:05:09.273ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
102772025-12-16T03:05:09.273ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 34.570878ms iteration = 73 status = {"config_updated":false}
102782025-12-16T03:05:09.278ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102792025-12-16T03:05:09.282ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
102802025-12-16T03:05:09.282ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
102812025-12-16T03:05:09.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
102822025-12-16T03:05:09.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102832025-12-16T03:05:09.287ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
102842025-12-16T03:05:09.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
102852025-12-16T03:05:09.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
102862025-12-16T03:05:09.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
102872025-12-16T03:05:09.290ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
102882025-12-16T03:05:09.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
102892025-12-16T03:05:09.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
102902025-12-16T03:05:09.292ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
102912025-12-16T03:05:09.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
102922025-12-16T03:05:09.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102932025-12-16T03:05:09.299ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
102942025-12-16T03:05:09.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
102952025-12-16T03:05:09.303ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
102962025-12-16T03:05:09.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
102972025-12-16T03:05:09.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
102982025-12-16T03:05:09.306ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
102992025-12-16T03:05:09.307ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
103002025-12-16T03:05:09.307ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
103012025-12-16T03:05:09.307ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103022025-12-16T03:05:09.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
103032025-12-16T03:05:09.309ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103042025-12-16T03:05:09.311ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
103052025-12-16T03:05:09.311ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
103062025-12-16T03:05:09.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
103072025-12-16T03:05:09.311ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103082025-12-16T03:05:09.312ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
103092025-12-16T03:05:09.312ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
103102025-12-16T03:05:09.312ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 73.35364ms iteration = 25 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
103112025-12-16T03:05:09.315ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103122025-12-16T03:05:09.317ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
103132025-12-16T03:05:09.317ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
103142025-12-16T03:05:09.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
103152025-12-16T03:05:09.318ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
103162025-12-16T03:05:09.318ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
103172025-12-16T03:05:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
103182025-12-16T03:05:09.320ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 81.595364ms iteration = 13 status = {"errors":[],"garbage_collect_requested":[]}
103192025-12-16T03:05:09.321ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
103202025-12-16T03:05:09.321ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
103212025-12-16T03:05:09.321ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-12-16 02:59:16.791641 UTC dns_group = internal generation = 1
103222025-12-16T03:05:09.321ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 149.295162ms iteration = 7 status = {"generation":1}
103232025-12-16T03:05:09.321ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103242025-12-16T03:05:09.322ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103252025-12-16T03:05:09.323ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
103262025-12-16T03:05:09.323ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
103272025-12-16T03:05:09.323ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 109.299394ms iteration = 7 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
103282025-12-16T03:05:09.323ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
103292025-12-16T03:05:09.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
103302025-12-16T03:05:09.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
103312025-12-16T03:05:09.326ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
103322025-12-16T03:05:09.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
103332025-12-16T03:05:09.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
103342025-12-16T03:05:09.326ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
103352025-12-16T03:05:09.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
103362025-12-16T03:05:09.327ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 88.284849ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
103372025-12-16T03:05:09.327ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 113.731831ms iteration = 4 status = {"nsuccess":5}
103382025-12-16T03:05:09.329ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103392025-12-16T03:05:09.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
103402025-12-16T03:05:09.333ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
103412025-12-16T03:05:09.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
103422025-12-16T03:05:09.335ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
103432025-12-16T03:05:09.335ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 96.265419ms iteration = 37 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
103442025-12-16T03:05:09.338ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
103452025-12-16T03:05:09.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
103462025-12-16T03:05:09.340ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103472025-12-16T03:05:09.342ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
103482025-12-16T03:05:09.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
103492025-12-16T03:05:09.344ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-12-16T02:59:05.020237Z, time_modified: 2025-12-16T02:59:05.020237Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }) }
103502025-12-16T03:05:09.344ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 172.411435ms iteration = 7 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]}},"default_endpoint":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
103512025-12-16T03:05:09.344ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
103522025-12-16T03:05:09.344ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
103532025-12-16T03:05:09.345ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 105.842981ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
103542025-12-16T03:05:09.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
103552025-12-16T03:05:09.346ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
103562025-12-16T03:05:09.346ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 107.829686ms iteration = 19 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
103572025-12-16T03:05:09.356ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
103582025-12-16T03:05:09.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
103592025-12-16T03:05:09.359ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
103602025-12-16T03:05:09.361ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103612025-12-16T03:05:09.364ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
103622025-12-16T03:05:09.364ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
103632025-12-16T03:05:09.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
103642025-12-16T03:05:09.369ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
103652025-12-16T03:05:09.369ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
103662025-12-16T03:05:09.369ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 130.379642ms iteration = 13 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
103672025-12-16T03:05:09.371ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 132.002213ms iteration = 13 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
103682025-12-16T03:05:09.402ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 163.214148ms iteration = 21 status = {"physical_disks_added":0}
103692025-12-16T03:05:09.413ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
103702025-12-16T03:05:09.413ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 174.160387ms iteration = 13 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
103712025-12-16T03:05:09.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
103722025-12-16T03:05:09.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
103732025-12-16T03:05:09.418ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103742025-12-16T03:05:09.422ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
103752025-12-16T03:05:09.422ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
103762025-12-16T03:05:09.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
103772025-12-16T03:05:09.423ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103782025-12-16T03:05:09.425ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
103792025-12-16T03:05:09.426ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
103802025-12-16T03:05:09.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
103812025-12-16T03:05:09.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
103822025-12-16T03:05:09.429ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
103832025-12-16T03:05:09.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
103842025-12-16T03:05:09.431ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
103852025-12-16T03:05:09.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
103862025-12-16T03:05:09.431ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103872025-12-16T03:05:09.434ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
103882025-12-16T03:05:09.434ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
103892025-12-16T03:05:09.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
103902025-12-16T03:05:09.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
103912025-12-16T03:05:09.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
103922025-12-16T03:05:09.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103932025-12-16T03:05:09.437ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103942025-12-16T03:05:09.441ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
103952025-12-16T03:05:09.441ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
103962025-12-16T03:05:09.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
103972025-12-16T03:05:09.441ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
103982025-12-16T03:05:09.441ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
103992025-12-16T03:05:09.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104002025-12-16T03:05:09.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104012025-12-16T03:05:09.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104022025-12-16T03:05:09.446ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a3b1ba8b-606a-418f-b5f7-9d063a1bf17e step component = DeployNexusRecords step id = Ensure
104032025-12-16T03:05:09.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
104042025-12-16T03:05:09.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
104052025-12-16T03:05:09.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104062025-12-16T03:05:09.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104072025-12-16T03:05:09.451ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
104082025-12-16T03:05:09.451ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
104092025-12-16T03:05:09.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
104102025-12-16T03:05:09.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
104112025-12-16T03:05:09.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
104122025-12-16T03:05:09.454ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
104132025-12-16T03:05:09.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104142025-12-16T03:05:09.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104152025-12-16T03:05:09.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104162025-12-16T03:05:09.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104172025-12-16T03:05:09.460ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
104182025-12-16T03:05:09.461ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
104192025-12-16T03:05:09.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104202025-12-16T03:05:09.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104212025-12-16T03:05:09.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104222025-12-16T03:05:09.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104232025-12-16T03:05:09.465ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
104242025-12-16T03:05:09.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
104252025-12-16T03:05:09.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104262025-12-16T03:05:09.470ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
104272025-12-16T03:05:09.470ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104282025-12-16T03:05:09.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104292025-12-16T03:05:09.472ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104302025-12-16T03:05:09.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104312025-12-16T03:05:09.476ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104322025-12-16T03:05:09.479ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
104332025-12-16T03:05:09.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104342025-12-16T03:05:09.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104352025-12-16T03:05:09.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104362025-12-16T03:05:09.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104372025-12-16T03:05:09.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104382025-12-16T03:05:09.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
104392025-12-16T03:05:09.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104402025-12-16T03:05:09.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104412025-12-16T03:05:09.490ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
104422025-12-16T03:05:09.490ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
104432025-12-16T03:05:09.491ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a3f8f53-15b1-4128-a594-1c5d1422f2e0", "content-length": "26", "date": "Tue, 16 Dec 2025 03:05:09 GMT"} })
104442025-12-16T03:05:09.491ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
104452025-12-16T03:05:09.493ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
104462025-12-16T03:05:09.494ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1317, tv_nsec: 106821113 } })
104472025-12-16T03:05:09.514ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a3b1ba8b-606a-418f-b5f7-9d063a1bf17e step component = SledAgent step id = Ensure
104482025-12-16T03:05:09.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104492025-12-16T03:05:09.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104502025-12-16T03:05:09.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104512025-12-16T03:05:09.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104522025-12-16T03:05:09.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104532025-12-16T03:05:09.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
104542025-12-16T03:05:09.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104552025-12-16T03:05:09.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104562025-12-16T03:05:09.528ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
104572025-12-16T03:05:09.529ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "ea0b9fc4-861f-4819-a682-3acd3284de16", "date": "Tue, 16 Dec 2025 03:05:09 GMT"} })
104582025-12-16T03:05:09.529ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
104592025-12-16T03:05:09.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104602025-12-16T03:05:09.532ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
104612025-12-16T03:05:09.532ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 300.429976ms iteration = 7 status = {"errors":[],"probes_by_sled":{"b1c76b61-acee-4be6-b582-973c938d6b6e":0}}
104622025-12-16T03:05:09.533ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
104632025-12-16T03:05:09.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104642025-12-16T03:05:09.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104652025-12-16T03:05:09.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104662025-12-16T03:05:09.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104672025-12-16T03:05:09.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104682025-12-16T03:05:09.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104692025-12-16T03:05:09.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104702025-12-16T03:05:09.546ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
104712025-12-16T03:05:09.546ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104722025-12-16T03:05:09.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104732025-12-16T03:05:09.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104742025-12-16T03:05:09.552ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
104752025-12-16T03:05:09.552ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104762025-12-16T03:05:09.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104772025-12-16T03:05:09.555ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104782025-12-16T03:05:09.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104792025-12-16T03:05:09.555ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104802025-12-16T03:05:09.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104812025-12-16T03:05:09.558ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104822025-12-16T03:05:09.561ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
104832025-12-16T03:05:09.562ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104842025-12-16T03:05:09.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104852025-12-16T03:05:09.563ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104862025-12-16T03:05:09.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104872025-12-16T03:05:09.565ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104882025-12-16T03:05:09.568ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
104892025-12-16T03:05:09.569ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104902025-12-16T03:05:09.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104912025-12-16T03:05:09.572ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104922025-12-16T03:05:09.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104932025-12-16T03:05:09.574ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104942025-12-16T03:05:09.577ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
104952025-12-16T03:05:09.577ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104962025-12-16T03:05:09.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104972025-12-16T03:05:09.578ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104982025-12-16T03:05:09.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104992025-12-16T03:05:09.580ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105002025-12-16T03:05:09.583ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
105012025-12-16T03:05:09.583ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105022025-12-16T03:05:09.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105032025-12-16T03:05:09.585ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105042025-12-16T03:05:09.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105052025-12-16T03:05:09.586ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105062025-12-16T03:05:09.587ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
105072025-12-16T03:05:09.588ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "72f60423-3749-454f-bb2d-41a98d584a70", "date": "Tue, 16 Dec 2025 03:05:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
105082025-12-16T03:05:09.588ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
105092025-12-16T03:05:09.590ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "e7a6783b-b0ab-4101-a678-85d55cb22965", "date": "Tue, 16 Dec 2025 03:05:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
105102025-12-16T03:05:09.590ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
105112025-12-16T03:05:09.591ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
105122025-12-16T03:05:09.591ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105132025-12-16T03:05:09.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105142025-12-16T03:05:09.592ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
105152025-12-16T03:05:09.592ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 420.325825ms iteration = 9 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
105162025-12-16T03:05:09.593ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105172025-12-16T03:05:09.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105182025-12-16T03:05:09.594ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
105192025-12-16T03:05:09.595ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 423.492395ms iteration = 13 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
105202025-12-16T03:05:09.595ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105212025-12-16T03:05:09.598ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
105222025-12-16T03:05:09.598ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105232025-12-16T03:05:09.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105242025-12-16T03:05:09.634ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
105252025-12-16T03:05:09.634ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
105262025-12-16T03:05:09.634ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "562e77ee-4054-4713-bf08-7187297a8ff1", "content-length": "26", "date": "Tue, 16 Dec 2025 03:05:09 GMT"} })
105272025-12-16T03:05:09.634ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
105282025-12-16T03:05:09.637ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
105292025-12-16T03:05:09.644ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
105302025-12-16T03:05:09.644ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
105312025-12-16T03:05:09.645ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 11c3b53b-d5f8-4fcd-9589-7b20c8495649 parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
105322025-12-16T03:05:09.645ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105332025-12-16T03:05:09.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105342025-12-16T03:05:09.646ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105352025-12-16T03:05:09.650ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
105362025-12-16T03:05:09.650ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105372025-12-16T03:05:09.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105382025-12-16T03:05:09.653ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
105392025-12-16T03:05:09.653ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
105402025-12-16T03:05:09.653ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 439.477717ms iteration = 17 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
105412025-12-16T03:05:09.653ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
105422025-12-16T03:05:09.653ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
105432025-12-16T03:05:09.654ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "1952a7f7-aaef-4ca6-828e-dc44ccbfe622", "content-length": "372", "date": "Tue, 16 Dec 2025 03:05:09 GMT"} })
105442025-12-16T03:05:09.654ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
105452025-12-16T03:05:09.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
105462025-12-16T03:05:09.656ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
105472025-12-16T03:05:09.656ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 442.538927ms iteration = 7 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
105482025-12-16T03:05:09.668ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
105492025-12-16T03:05:09.668ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
105502025-12-16T03:05:09.669ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "52aad5a9-8f54-4899-983e-55731e66c04a", "date": "Tue, 16 Dec 2025 03:05:09 GMT"} })
105512025-12-16T03:05:09.669ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
105522025-12-16T03:05:09.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
105532025-12-16T03:05:09.671ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
105542025-12-16T03:05:09.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
105552025-12-16T03:05:09.675ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
105562025-12-16T03:05:09.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
105572025-12-16T03:05:09.678ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
105582025-12-16T03:05:09.678ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
105592025-12-16T03:05:09.679ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "196a6fbc-49aa-4390-b212-d7d7670df401", "date": "Tue, 16 Dec 2025 03:05:09 GMT"} })
105602025-12-16T03:05:09.681ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 442.254803ms iteration = 19 status = {}
105612025-12-16T03:05:09.681ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 20 reason = Dependency
105622025-12-16T03:05:09.681ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
105632025-12-16T03:05:09.681ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
105642025-12-16T03:05:09.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
105652025-12-16T03:05:09.688ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105662025-12-16T03:05:09.690ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 518.101874ms iteration = 13 status = {}
105672025-12-16T03:05:09.691ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
105682025-12-16T03:05:09.691ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
105692025-12-16T03:05:09.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
105702025-12-16T03:05:09.693ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105712025-12-16T03:05:09.694ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
105722025-12-16T03:05:09.697ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
105732025-12-16T03:05:09.697ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
105742025-12-16T03:05:09.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
105752025-12-16T03:05:09.698ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
105762025-12-16T03:05:09.702ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
105772025-12-16T03:05:09.705ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb0c1c45-794c-44ca-bd58-17305d772f88", "content-length": "691", "date": "Tue, 16 Dec 2025 03:05:09 GMT"} })
105782025-12-16T03:05:09.705ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
105792025-12-16T03:05:09.705ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
105802025-12-16T03:05:09.705ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
105812025-12-16T03:05:09.707ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "25165b6e-6e77-4313-bdf9-dd679f3663b5", "date": "Tue, 16 Dec 2025 03:05:09 GMT"} }) sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
105822025-12-16T03:05:09.707ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 468.196841ms iteration = 13 status = {}
105832025-12-16T03:05:09.709ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a3b1ba8b-606a-418f-b5f7-9d063a1bf17e step component = FirewallRules step id = Ensure
105842025-12-16T03:05:09.709ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
105852025-12-16T03:05:09.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
105862025-12-16T03:05:09.711ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
105872025-12-16T03:05:09.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
105882025-12-16T03:05:09.713ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
105892025-12-16T03:05:09.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
105902025-12-16T03:05:09.715ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
105912025-12-16T03:05:09.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
105922025-12-16T03:05:09.717ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
105932025-12-16T03:05:09.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
105942025-12-16T03:05:09.719ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
105952025-12-16T03:05:09.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
105962025-12-16T03:05:09.720ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105972025-12-16T03:05:09.725ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
105982025-12-16T03:05:09.725ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
105992025-12-16T03:05:09.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106002025-12-16T03:05:09.726ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106012025-12-16T03:05:09.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106022025-12-16T03:05:09.729ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106032025-12-16T03:05:09.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106042025-12-16T03:05:09.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106052025-12-16T03:05:09.734ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1317, tv_nsec: 347088807 } })
106062025-12-16T03:05:09.735ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
106072025-12-16T03:05:09.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106082025-12-16T03:05:09.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106092025-12-16T03:05:09.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106102025-12-16T03:05:09.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106112025-12-16T03:05:09.739ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106122025-12-16T03:05:09.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106132025-12-16T03:05:09.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106142025-12-16T03:05:09.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106152025-12-16T03:05:09.742ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106162025-12-16T03:05:09.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106172025-12-16T03:05:09.744ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106182025-12-16T03:05:09.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106192025-12-16T03:05:09.745ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106202025-12-16T03:05:09.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106212025-12-16T03:05:09.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106222025-12-16T03:05:09.748ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
106232025-12-16T03:05:09.748ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "921a603d-2117-44b3-b4e7-5a9c0da80793", "date": "Tue, 16 Dec 2025 03:05:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
106242025-12-16T03:05:09.748ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
106252025-12-16T03:05:09.750ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "42cef698-ccfe-478f-b2dd-ef04f2e6f9e3", "date": "Tue, 16 Dec 2025 03:05:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
106262025-12-16T03:05:09.750ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
106272025-12-16T03:05:09.751ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
106282025-12-16T03:05:09.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106292025-12-16T03:05:09.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106302025-12-16T03:05:09.751ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
106312025-12-16T03:05:09.751ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
106322025-12-16T03:05:09.752ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "ab0e4671-cef6-43a7-94d3-37498881bc3b", "date": "Tue, 16 Dec 2025 03:05:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
106332025-12-16T03:05:09.752ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
106342025-12-16T03:05:09.752ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8bead77a-4b33-485d-b23f-4af914505955", "content-length": "372", "date": "Tue, 16 Dec 2025 03:05:09 GMT"} })
106352025-12-16T03:05:09.752ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
106362025-12-16T03:05:09.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
106372025-12-16T03:05:09.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106382025-12-16T03:05:09.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106392025-12-16T03:05:09.754ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
106402025-12-16T03:05:09.754ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 581.909793ms iteration = 9 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
106412025-12-16T03:05:09.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106422025-12-16T03:05:09.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106432025-12-16T03:05:09.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106442025-12-16T03:05:09.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106452025-12-16T03:05:09.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106462025-12-16T03:05:09.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106472025-12-16T03:05:09.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106482025-12-16T03:05:09.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106492025-12-16T03:05:09.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106502025-12-16T03:05:09.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106512025-12-16T03:05:09.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106522025-12-16T03:05:09.769ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
106532025-12-16T03:05:09.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106542025-12-16T03:05:09.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106552025-12-16T03:05:09.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106562025-12-16T03:05:09.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106572025-12-16T03:05:09.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106582025-12-16T03:05:09.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106592025-12-16T03:05:09.773ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106602025-12-16T03:05:09.774ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1317, tv_nsec: 387548520 } })
106612025-12-16T03:05:09.778ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
106622025-12-16T03:05:09.778ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106632025-12-16T03:05:09.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106642025-12-16T03:05:09.783ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106652025-12-16T03:05:09.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106662025-12-16T03:05:09.784ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106672025-12-16T03:05:09.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106682025-12-16T03:05:09.786ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106692025-12-16T03:05:09.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106702025-12-16T03:05:09.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106712025-12-16T03:05:09.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106722025-12-16T03:05:09.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106732025-12-16T03:05:09.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106742025-12-16T03:05:09.791ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106752025-12-16T03:05:09.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106762025-12-16T03:05:09.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106772025-12-16T03:05:09.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106782025-12-16T03:05:09.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106792025-12-16T03:05:09.800ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
106802025-12-16T03:05:09.800ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106812025-12-16T03:05:09.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106822025-12-16T03:05:09.802ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106832025-12-16T03:05:09.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106842025-12-16T03:05:09.803ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106852025-12-16T03:05:09.803ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
106862025-12-16T03:05:09.803ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
106872025-12-16T03:05:09.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106882025-12-16T03:05:09.804ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "fcd3b946-6b1f-4f86-96dc-180116a574d5", "date": "Tue, 16 Dec 2025 03:05:09 GMT"} })
106892025-12-16T03:05:09.804ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
106902025-12-16T03:05:09.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
106912025-12-16T03:05:09.805ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106922025-12-16T03:05:09.806ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
106932025-12-16T03:05:09.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
106942025-12-16T03:05:09.807ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
106952025-12-16T03:05:09.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
106962025-12-16T03:05:09.808ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
106972025-12-16T03:05:09.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
106982025-12-16T03:05:09.809ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
106992025-12-16T03:05:09.809ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
107002025-12-16T03:05:09.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
107012025-12-16T03:05:09.810ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
107022025-12-16T03:05:09.810ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107032025-12-16T03:05:09.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107042025-12-16T03:05:09.811ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
107052025-12-16T03:05:09.811ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
107062025-12-16T03:05:09.811ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "98c29b64-56ca-4577-9666-6ab428b615c3", "date": "Tue, 16 Dec 2025 03:05:09 GMT"} })
107072025-12-16T03:05:09.813ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 132.161785ms iteration = 20 status = {}
107082025-12-16T03:05:09.816ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], Name("nexus"): [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
107092025-12-16T03:05:09.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107102025-12-16T03:05:09.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107112025-12-16T03:05:09.817ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
107122025-12-16T03:05:09.817ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
107132025-12-16T03:05:09.818ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107142025-12-16T03:05:09.818ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
107152025-12-16T03:05:09.818ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
107162025-12-16T03:05:09.818ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
107172025-12-16T03:05:09.818ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
107182025-12-16T03:05:09.818ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
107192025-12-16T03:05:09.818ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
107202025-12-16T03:05:09.818ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
107212025-12-16T03:05:09.818ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
107222025-12-16T03:05:09.819ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
107232025-12-16T03:05:09.819ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
107242025-12-16T03:05:09.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
107252025-12-16T03:05:09.820ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
107262025-12-16T03:05:09.820ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107272025-12-16T03:05:09.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107282025-12-16T03:05:09.821ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
107292025-12-16T03:05:09.821ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
107302025-12-16T03:05:09.822ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
107312025-12-16T03:05:09.822ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
107322025-12-16T03:05:09.822ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
107332025-12-16T03:05:09.897ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), time_created: 2025-12-16T02:59:15.195809Z, time_modified: 2025-12-16T02:59:15.195809Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
107342025-12-16T03:05:09.897ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
107352025-12-16T03:05:09.929ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107362025-12-16T03:05:09.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107372025-12-16T03:05:09.931ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107382025-12-16T03:05:09.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107392025-12-16T03:05:09.933ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107402025-12-16T03:05:09.936ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
107412025-12-16T03:05:09.936ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
107422025-12-16T03:05:09.937ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "c6679b71-6ae0-4ee7-8248-1419c283c30d", "date": "Tue, 16 Dec 2025 03:05:09 GMT"} })
107432025-12-16T03:05:09.939ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
107442025-12-16T03:05:09.939ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a3b1ba8b-606a-418f-b5f7-9d063a1bf17e step component = Dns step id = Ensure
107452025-12-16T03:05:09.939ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
107462025-12-16T03:05:09.939ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107472025-12-16T03:05:09.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107482025-12-16T03:05:09.940ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "ccfce704-3347-4d2a-9cce-ecad7d27c03d", "date": "Tue, 16 Dec 2025 03:05:09 GMT"} })
107492025-12-16T03:05:09.940ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
107502025-12-16T03:05:09.940ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
107512025-12-16T03:05:09.940ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107522025-12-16T03:05:09.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107532025-12-16T03:05:09.940ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4c8f323-7e1f-432f-8c29-d943d637236e", "content-length": "682", "date": "Tue, 16 Dec 2025 03:05:09 GMT"} })
107542025-12-16T03:05:09.940ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
107552025-12-16T03:05:09.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
107562025-12-16T03:05:09.942ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107572025-12-16T03:05:09.944ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
107582025-12-16T03:05:09.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
107592025-12-16T03:05:09.947ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
107602025-12-16T03:05:09.947ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
107612025-12-16T03:05:09.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
107622025-12-16T03:05:09.947ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
107632025-12-16T03:05:09.947ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
107642025-12-16T03:05:09.947ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
107652025-12-16T03:05:09.947ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107662025-12-16T03:05:09.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107672025-12-16T03:05:09.949ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
107682025-12-16T03:05:09.949ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107692025-12-16T03:05:09.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
107702025-12-16T03:05:09.952ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
107712025-12-16T03:05:09.952ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107722025-12-16T03:05:09.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107732025-12-16T03:05:09.954ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
107742025-12-16T03:05:09.954ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
107752025-12-16T03:05:09.954ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107762025-12-16T03:05:09.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107772025-12-16T03:05:09.955ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 716.507838ms iteration = 13 status = {}
107782025-12-16T03:05:09.956ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107792025-12-16T03:05:09.960ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
107802025-12-16T03:05:09.960ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107812025-12-16T03:05:09.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107822025-12-16T03:05:09.964ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
107832025-12-16T03:05:09.964ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
107842025-12-16T03:05:09.964ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
107852025-12-16T03:05:09.964ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107862025-12-16T03:05:09.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107872025-12-16T03:05:09.964ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107882025-12-16T03:05:09.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107892025-12-16T03:05:09.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107902025-12-16T03:05:09.970ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
107912025-12-16T03:05:09.971ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
107922025-12-16T03:05:09.971ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
107932025-12-16T03:05:09.971ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
107942025-12-16T03:05:09.971ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107952025-12-16T03:05:09.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107962025-12-16T03:05:09.973ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107972025-12-16T03:05:09.976ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
107982025-12-16T03:05:09.976ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107992025-12-16T03:05:09.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108002025-12-16T03:05:09.978ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
108012025-12-16T03:05:09.978ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
108022025-12-16T03:05:09.978ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108032025-12-16T03:05:09.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108042025-12-16T03:05:09.980ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108052025-12-16T03:05:09.984ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
108062025-12-16T03:05:09.984ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108072025-12-16T03:05:09.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108082025-12-16T03:05:09.985ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
108092025-12-16T03:05:09.985ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
108102025-12-16T03:05:09.985ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108112025-12-16T03:05:09.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108122025-12-16T03:05:09.987ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108132025-12-16T03:05:09.989ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
108142025-12-16T03:05:09.989ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108152025-12-16T03:05:09.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108162025-12-16T03:05:09.991ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108172025-12-16T03:05:09.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108182025-12-16T03:05:09.992ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108192025-12-16T03:05:09.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108202025-12-16T03:05:09.993ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108212025-12-16T03:05:09.996ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 2
108222025-12-16T03:05:09.996ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221
108232025-12-16T03:05:09.997ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
108242025-12-16T03:05:09.999ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
108252025-12-16T03:05:09.999ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
108262025-12-16T03:05:09.999ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a3b1ba8b-606a-418f-b5f7-9d063a1bf17e step component = OmicronZones step id = Cleanup
108272025-12-16T03:05:09.999ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a3b1ba8b-606a-418f-b5f7-9d063a1bf17e step component = OmicronZones step id = Cleanup
108282025-12-16T03:05:09.999ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a3b1ba8b-606a-418f-b5f7-9d063a1bf17e step component = PhysicalDisks step id = Cleanup
108292025-12-16T03:05:09.999ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a3b1ba8b-606a-418f-b5f7-9d063a1bf17e step component = Clickhouse step id = Ensure
108302025-12-16T03:05:09.999ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a3b1ba8b-606a-418f-b5f7-9d063a1bf17e step component = Clickhouse step id = Ensure
108312025-12-16T03:05:10.112ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
108322025-12-16T03:05:10.112ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:63981
108332025-12-16T03:05:10.112ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:63981 req_id = a70045e7-5573-405b-b88e-d439f0c90eb0 uri = /metrics/producers version = 1.0.0
108342025-12-16T03:05:10.112ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:63981 req_id = a70045e7-5573-405b-b88e-d439f0c90eb0 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
108352025-12-16T03:05:10.112ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:63981 req_id = a70045e7-5573-405b-b88e-d439f0c90eb0 resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
108362025-12-16T03:05:10.116ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
108372025-12-16T03:05:10.121ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "408d7102-43a2-47ee-9b8f-26a42ca5493e", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
108382025-12-16T03:05:10.124ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a3b1ba8b-606a-418f-b5f7-9d063a1bf17e step component = SupportBundles step id = Cleanup
108392025-12-16T03:05:10.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108402025-12-16T03:05:10.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108412025-12-16T03:05:10.126ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108422025-12-16T03:05:10.128ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
108432025-12-16T03:05:10.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108442025-12-16T03:05:10.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108452025-12-16T03:05:10.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108462025-12-16T03:05:10.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108472025-12-16T03:05:10.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108482025-12-16T03:05:10.134ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
108492025-12-16T03:05:10.139ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a3b1ba8b-606a-418f-b5f7-9d063a1bf17e step component = OmicronZones step id = Cleanup
108502025-12-16T03:05:10.139ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
108512025-12-16T03:05:10.139ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
108522025-12-16T03:05:10.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108532025-12-16T03:05:10.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108542025-12-16T03:05:10.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108552025-12-16T03:05:10.142ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
108562025-12-16T03:05:10.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108572025-12-16T03:05:10.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108582025-12-16T03:05:10.184ZINFOnexus: assigned collector to new producer collector_id = e5b81e4f-90b9-43ba-ba56-584fd121e601 file = nexus/src/app/oximeter.rs:76 producer_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
108592025-12-16T03:05:10.184ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 72356 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:63981 req_id = a70045e7-5573-405b-b88e-d439f0c90eb0 response_code = 201 uri = /metrics/producers
108602025-12-16T03:05:10.185ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "a70045e7-5573-405b-b88e-d439f0c90eb0", "content-length": "41", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
108612025-12-16T03:05:10.186ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
108622025-12-16T03:05:10.186ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(a91734db-9ac7-4793-9c32-ddbb21f8d986 (blueprint), false)
108632025-12-16T03:05:10.186ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a3b1ba8b-606a-418f-b5f7-9d063a1bf17e step component = Cockroach step id = Ensure
108642025-12-16T03:05:10.186ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108652025-12-16T03:05:10.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108662025-12-16T03:05:10.187ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
108672025-12-16T03:05:10.208ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
108682025-12-16T03:05:10.208ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = a3b1ba8b-606a-418f-b5f7-9d063a1bf17e step component = MgsUpdates step id = Ensure
108692025-12-16T03:05:10.208ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 8 reason = Dependency
108702025-12-16T03:05:10.208ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 20 reason = Dependency
108712025-12-16T03:05:10.208ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
108722025-12-16T03:05:10.208ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
108732025-12-16T03:05:10.208ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.036423061s iteration = 8 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"05d20ffa-30da-4ce4-9dd0-2e6456102989","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"05d20ffa-30da-4ce4-9dd0-2e6456102989","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":4240,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":71565317,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":71565277,"secs":0}},"event_index":1,"execution_id":"05d20ffa-30da-4ce4-9dd0-2e6456102989","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":71582917,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":202707978,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":202707927,"secs":0}},"event_index":2,"execution_id":"05d20ffa-30da-4ce4-9dd0-2e6456102989","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":274313164,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":68040392,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":68040342,"secs":0}},"event_index":3,"execution_id":"05d20ffa-30da-4ce4-9dd0-2e6456102989","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":342396347,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":195086352,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":195086302,"secs":0}},"event_index":4,"execution_id":"05d20ffa-30da-4ce4-9dd0-2e6456102989","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":537503069,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":229710071,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":229710041,"secs":0}},"event_index":5,"execution_id":"05d20ffa-30da-4ce4-9dd0-2e6456102989","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":767227250,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":59473734,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":59473694,"secs":0}},"event_index":6,"execution_id":"05d20ffa-30da-4ce4-9dd0-2e6456102989","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":826713464,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":6800,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":6760,"secs":0}},"event_index":7,"execution_id":"05d20ffa-30da-4ce4-9dd0-2e6456102989","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":826739134,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3640,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":3600,"secs":0}},"event_index":8,"execution_id":"05d20ffa-30da-4ce4-9dd0-2e6456102989","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":826746834,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5371,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5331,"secs":0}},"event_index":9,"execution_id":"05d20ffa-30da-4ce4-9dd0-2e6456102989","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":826760264,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2580,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":2531,"secs":0}},"event_index":10,"execution_id":"05d20ffa-30da-4ce4-9dd0-2e6456102989","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":826765084,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":124923912,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":124923882,"secs":0}},"event_index":11,"execution_id":"05d20ffa-30da-4ce4-9dd0-2e6456102989","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":951691577,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":14963719,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":14963679,"secs":0}},"event_index":12,"execution_id":"05d20ffa-30da-4ce4-9dd0-2e6456102989","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":966670997,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":47331340,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":47331310,"secs":0}},"event_index":13,"execution_id":"05d20ffa-30da-4ce4-9dd0-2e6456102989","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":14013097,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":22085350,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":22085310,"secs":0}},"event_index":14,"execution_id":"05d20ffa-30da-4ce4-9dd0-2e6456102989","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":36113467,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5850,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5810,"secs":0}},"event_index":15,"execution_id":"05d20ffa-30da-4ce4-9dd0-2e6456102989","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":36130017,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986"}
108742025-12-16T03:05:10.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
108752025-12-16T03:05:10.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
108762025-12-16T03:05:10.210ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108772025-12-16T03:05:10.210ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108782025-12-16T03:05:10.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
108792025-12-16T03:05:10.213ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
108802025-12-16T03:05:10.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
108812025-12-16T03:05:10.213ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
108822025-12-16T03:05:10.213ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
108832025-12-16T03:05:10.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
108842025-12-16T03:05:10.215ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
108852025-12-16T03:05:10.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
108862025-12-16T03:05:10.217ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108872025-12-16T03:05:10.219ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
108882025-12-16T03:05:10.219ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
108892025-12-16T03:05:10.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
108902025-12-16T03:05:10.221ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
108912025-12-16T03:05:10.221ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 12.57216ms iteration = 20 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
108922025-12-16T03:05:10.240ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(58e21b5f-57d6-4b9f-861f-24cabd63d443, 0), (b48a1e8c-4c75-4571-b0b8-834af8e60fa2, 0), (89379007-adea-404b-abcf-9846c3901b08, 0), (90409fae-ab64-4abc-8dde-084b1a5d45d1, 0)] collection_id = 58e21b5f-57d6-4b9f-861f-24cabd63d443
108932025-12-16T03:05:10.240ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
108942025-12-16T03:05:10.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
108952025-12-16T03:05:10.408ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 58e21b5f-57d6-4b9f-861f-24cabd63d443 file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
108962025-12-16T03:05:10.408ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
108972025-12-16T03:05:10.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
108982025-12-16T03:05:10.414ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(b48a1e8c-4c75-4571-b0b8-834af8e60fa2, 0), (89379007-adea-404b-abcf-9846c3901b08, 0), (90409fae-ab64-4abc-8dde-084b1a5d45d1, 0), (5252caf0-f0b7-49df-83a4-ee1dc3a503fd, 0)] collection_id = b48a1e8c-4c75-4571-b0b8-834af8e60fa2
108992025-12-16T03:05:10.414ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109002025-12-16T03:05:10.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
109012025-12-16T03:05:10.619ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = b48a1e8c-4c75-4571-b0b8-834af8e60fa2 file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
109022025-12-16T03:05:10.619ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109032025-12-16T03:05:10.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
109042025-12-16T03:05:10.623ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(89379007-adea-404b-abcf-9846c3901b08, 0), (90409fae-ab64-4abc-8dde-084b1a5d45d1, 0), (5252caf0-f0b7-49df-83a4-ee1dc3a503fd, 0)]
109052025-12-16T03:05:10.623ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1318, tv_nsec: 236501256 } })
109062025-12-16T03:05:10.656ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("20488683-5778-4090-b36e-e6f0d80a682b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("35dbaad1-8536-44a0-8a0a-35d7c39dcfbb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b57fa226-2428-4a54-b85d-60409e28b484.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("d812a3e2-48ae-4812-a1cc-7eb7623e4e00.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::5)) }], valid_until: Instant { tv_sec: 1318, tv_nsec: 269405813 } })
109072025-12-16T03:05:10.657ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
109082025-12-16T03:05:10.657ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
109092025-12-16T03:05:10.657ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
109102025-12-16T03:05:10.657ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca3fb389-ae8e-455f-98ba-c0943e23ac33", "content-length": "476", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109112025-12-16T03:05:10.657ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
109122025-12-16T03:05:10.658ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f68cba6-a0d8-427a-9016-5e5c731fe512", "content-length": "734", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109132025-12-16T03:05:10.658ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
109142025-12-16T03:05:10.658ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff336cde-d0d1-467b-9aed-a5a1cbc00deb", "content-length": "111", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109152025-12-16T03:05:10.658ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
109162025-12-16T03:05:10.658ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0db25111-d5f3-4b10-9864-cb920deb0c60", "content-length": "111", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109172025-12-16T03:05:10.658ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
109182025-12-16T03:05:10.659ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c537ccb-e1d7-4ec9-af00-e309fb9a0050", "content-length": "163", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109192025-12-16T03:05:10.659ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
109202025-12-16T03:05:10.659ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "96edd3fd-94a1-42b5-8a83-531512bd037b", "content-length": "163", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109212025-12-16T03:05:10.659ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
109222025-12-16T03:05:10.660ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "28514179-bced-4ce4-99d0-dc1bb20bad41", "content-length": "166", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109232025-12-16T03:05:10.660ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
109242025-12-16T03:05:10.660ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e28f710-b7fa-43ec-8336-73da4fea5300", "content-length": "166", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109252025-12-16T03:05:10.660ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
109262025-12-16T03:05:10.660ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "749e3455-8468-459f-ad84-787d316ed70c", "content-length": "702", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109272025-12-16T03:05:10.660ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
109282025-12-16T03:05:10.660ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "27012e6c-2e36-4495-b548-e9001df35298", "content-length": "727", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109292025-12-16T03:05:10.661ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
109302025-12-16T03:05:10.661ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7abc9223-17fa-4c83-a485-444b11e04150", "content-length": "729", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109312025-12-16T03:05:10.661ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
109322025-12-16T03:05:10.661ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5fef0f6c-33cd-4108-9b58-68324a0ddc8f", "content-length": "728", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109332025-12-16T03:05:10.661ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
109342025-12-16T03:05:10.661ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f2977c3-c71a-4114-9637-8472191a661c", "content-length": "722", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109352025-12-16T03:05:10.661ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
109362025-12-16T03:05:10.661ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "33b542c1-a505-4129-b0d2-7bf77189544a", "content-length": "10", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109372025-12-16T03:05:10.661ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
109382025-12-16T03:05:10.661ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5363c3c6-bbb6-4232-9d15-ad2c481fa6e6", "content-length": "144", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109392025-12-16T03:05:10.662ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
109402025-12-16T03:05:10.662ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "122fed3c-e153-42b2-8371-d304d96a9d23", "content-length": "147", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109412025-12-16T03:05:10.662ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
109422025-12-16T03:05:10.662ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "51f688b6-f2c5-4a0c-9e5a-801e3aeca5f5", "content-length": "109", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109432025-12-16T03:05:10.662ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
109442025-12-16T03:05:10.662ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1dc521c6-0576-4264-a95d-d9896761c6bb", "content-length": "109", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109452025-12-16T03:05:10.662ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
109462025-12-16T03:05:10.663ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "87705cdc-cd58-4ed5-8e49-1514cdf43b8d", "content-length": "163", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109472025-12-16T03:05:10.663ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
109482025-12-16T03:05:10.663ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e700bbea-ffd7-4b49-a17c-2af5ec272342", "content-length": "163", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109492025-12-16T03:05:10.663ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
109502025-12-16T03:05:10.664ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c819767-d47a-425a-b0c7-25e5fb5a45c1", "content-length": "166", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109512025-12-16T03:05:10.664ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
109522025-12-16T03:05:10.664ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9cd12087-1d3d-4ad5-acf0-35a9114e06c3", "content-length": "166", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109532025-12-16T03:05:10.664ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
109542025-12-16T03:05:10.664ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2544a67-33db-49ee-a170-86bd5aa0df4c", "content-length": "702", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109552025-12-16T03:05:10.664ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
109562025-12-16T03:05:10.664ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8abaf0db-1880-4363-ae1e-907dbcf88cc7", "content-length": "727", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109572025-12-16T03:05:10.664ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
109582025-12-16T03:05:10.665ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d865c64d-efaf-4780-937e-77f3a7d26b8e", "content-length": "729", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109592025-12-16T03:05:10.665ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
109602025-12-16T03:05:10.665ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e51d0342-edd1-4048-8e22-b4b9e79f433b", "content-length": "728", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109612025-12-16T03:05:10.665ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109622025-12-16T03:05:10.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
109632025-12-16T03:05:10.667ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109642025-12-16T03:05:10.669ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
109652025-12-16T03:05:10.670ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109662025-12-16T03:05:10.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
109672025-12-16T03:05:10.671ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109682025-12-16T03:05:10.674ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
109692025-12-16T03:05:10.674ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109702025-12-16T03:05:10.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
109712025-12-16T03:05:10.706ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
109722025-12-16T03:05:10.706ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
109732025-12-16T03:05:10.761ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "3bf5e73d-f31b-4e08-bd75-03763c6b3501", "content-length": "95612", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109742025-12-16T03:05:10.762ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
109752025-12-16T03:05:10.762ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
109762025-12-16T03:05:10.762ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
109772025-12-16T03:05:10.762ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
109782025-12-16T03:05:10.762ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
109792025-12-16T03:05:10.762ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
109802025-12-16T03:05:10.762ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
109812025-12-16T03:05:10.762ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
109822025-12-16T03:05:10.763ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1cf78775-cb1e-4709-b735-198bc478f90c", "content-length": "64", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109832025-12-16T03:05:10.763ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1f483b8-9bb4-491e-907f-b8bf33bb179c", "content-length": "64", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109842025-12-16T03:05:10.763ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
109852025-12-16T03:05:10.763ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
109862025-12-16T03:05:10.763ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "da1112b7-07fe-499a-85d7-d6437a6ce300", "content-length": "64", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109872025-12-16T03:05:10.763ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
109882025-12-16T03:05:10.763ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fcbddd2a-08a8-42c9-9378-6e9563aec315", "content-length": "64", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109892025-12-16T03:05:10.763ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
109902025-12-16T03:05:10.763ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "578b0967-c0ef-441f-9c8c-226252bc26f3", "content-length": "64", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109912025-12-16T03:05:10.763ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
109922025-12-16T03:05:10.804ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "392ceeeb-ee5f-4931-8760-4fd0099624b0", "content-length": "398308", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109932025-12-16T03:05:10.804ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "d96ea0ff-6d18-4a54-a6d6-859620370a28", "content-length": "400841", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109942025-12-16T03:05:10.804ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "de4a50bb-8e2f-4523-bfe4-e48f1cba688c", "content-length": "398213", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109952025-12-16T03:05:10.806ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "9046e160-898d-47b4-960e-0cece0c76999", "content-length": "400181", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109962025-12-16T03:05:10.807ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2e29425-7e4b-46fe-b24a-d3ea90921bcb", "content-length": "429187", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
109972025-12-16T03:05:10.810ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
109982025-12-16T03:05:10.810ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
109992025-12-16T03:05:10.810ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
110002025-12-16T03:05:10.810ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
110012025-12-16T03:05:10.810ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
110022025-12-16T03:05:10.842ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c (omicron_zone)
110032025-12-16T03:05:10.842ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
110042025-12-16T03:05:10.846ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "c636f837-c6d7-4ac5-a16c-3c028ac74df8", "content-length": "124", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
110052025-12-16T03:05:10.847ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
110062025-12-16T03:05:10.942ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
110072025-12-16T03:05:10.943ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
110082025-12-16T03:05:10.943ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "2825dc3f-0733-48ab-9008-805cd43f8ce8", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
110092025-12-16T03:05:10.943ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
110102025-12-16T03:05:10.945ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
110112025-12-16T03:05:10.945ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
110122025-12-16T03:05:10.945ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "78259f46-afec-4819-85b0-711a066d0e66", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
110132025-12-16T03:05:10.946ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
110142025-12-16T03:05:10.947ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
110152025-12-16T03:05:10.947ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
110162025-12-16T03:05:10.947ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ead7c910-b157-4dac-860a-37fe0021fc19", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:05:10 GMT"} })
110172025-12-16T03:05:10.947ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
110182025-12-16T03:05:10.949ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
110192025-12-16T03:05:10.949ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
110202025-12-16T03:05:10.951ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
110212025-12-16T03:05:10.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
110222025-12-16T03:05:10.953ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110232025-12-16T03:05:10.956ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
110242025-12-16T03:05:10.956ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
110252025-12-16T03:05:10.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
110262025-12-16T03:05:11.062ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 3c35439c-8f26-4169-b227-d3e30abe5b3b file = nexus/db-queries/src/db/datastore/inventory.rs:1613
110272025-12-16T03:05:11.062ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 3c35439c-8f26-4169-b227-d3e30abe5b3b time_started = 2025-12-16 03:05:10.657315 UTC
110282025-12-16T03:05:11.062ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 23 reason = Dependency
110292025-12-16T03:05:11.062ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
110302025-12-16T03:05:11.062ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 33 reason = Dependency
110312025-12-16T03:05:11.062ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 17.871\u{b5}s iteration = 23 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
110322025-12-16T03:05:11.062ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
110332025-12-16T03:05:11.062ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 853.660496ms iteration = 8 status = {"collection_id":"3c35439c-8f26-4169-b227-d3e30abe5b3b","time_done":"2025-12-16 03:05:10.949695 UTC","time_started":"2025-12-16 03:05:10.657315 UTC"}
110342025-12-16T03:05:11.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
110352025-12-16T03:05:11.064ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110362025-12-16T03:05:11.064ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 332 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 0cfa0ff4-5f5d-4ae9-90eb-4d7859092ed4 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
110372025-12-16T03:05:11.068ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
110382025-12-16T03:05:11.068ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
110392025-12-16T03:05:11.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
110402025-12-16T03:05:11.276ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
110412025-12-16T03:05:11.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
110422025-12-16T03:05:11.326ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 264.494881ms iteration = 33 status = {"Loaded":{"collection_id":"b6c51ddf-9f27-432f-9d24-9fbc42795d0c","time_loaded":"2025-12-16T03:05:11.062187129Z","time_started":"2025-12-16T03:05:10.667073Z"}}
110432025-12-16T03:05:11.326ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 22 reason = Dependency
110442025-12-16T03:05:11.326ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 11 reason = Dependency
110452025-12-16T03:05:11.326ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 24 reason = Dependency
110462025-12-16T03:05:11.326ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
110472025-12-16T03:05:11.326ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 18 reason = Dependency
110482025-12-16T03:05:11.326ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
110492025-12-16T03:05:11.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
110502025-12-16T03:05:11.326ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 20.06\u{b5}s iteration = 24 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
110512025-12-16T03:05:11.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
110522025-12-16T03:05:11.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
110532025-12-16T03:05:11.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
110542025-12-16T03:05:11.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
110552025-12-16T03:05:11.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110562025-12-16T03:05:11.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110572025-12-16T03:05:11.328ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110582025-12-16T03:05:11.331ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
110592025-12-16T03:05:11.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
110602025-12-16T03:05:11.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
110612025-12-16T03:05:11.332ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
110622025-12-16T03:05:11.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
110632025-12-16T03:05:11.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
110642025-12-16T03:05:11.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
110652025-12-16T03:05:11.332ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
110662025-12-16T03:05:11.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
110672025-12-16T03:05:11.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110682025-12-16T03:05:11.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110692025-12-16T03:05:11.334ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
110702025-12-16T03:05:11.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
110712025-12-16T03:05:11.335ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
110722025-12-16T03:05:11.335ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
110732025-12-16T03:05:11.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
110742025-12-16T03:05:11.335ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110752025-12-16T03:05:11.336ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
110762025-12-16T03:05:11.336ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
110772025-12-16T03:05:11.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
110782025-12-16T03:05:11.338ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
110792025-12-16T03:05:11.338ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
110802025-12-16T03:05:11.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
110812025-12-16T03:05:11.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
110822025-12-16T03:05:11.338ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
110832025-12-16T03:05:11.338ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
110842025-12-16T03:05:11.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
110852025-12-16T03:05:11.339ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110862025-12-16T03:05:11.339ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110872025-12-16T03:05:11.341ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
110882025-12-16T03:05:11.341ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
110892025-12-16T03:05:11.341ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
110902025-12-16T03:05:11.341ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
110912025-12-16T03:05:11.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
110922025-12-16T03:05:11.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
110932025-12-16T03:05:11.343ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110942025-12-16T03:05:11.343ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110952025-12-16T03:05:11.346ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
110962025-12-16T03:05:11.346ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
110972025-12-16T03:05:11.346ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
110982025-12-16T03:05:11.346ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
110992025-12-16T03:05:11.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111002025-12-16T03:05:11.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
111012025-12-16T03:05:11.349ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
111022025-12-16T03:05:11.349ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
111032025-12-16T03:05:11.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
111042025-12-16T03:05:11.349ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111052025-12-16T03:05:11.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111062025-12-16T03:05:11.351ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111072025-12-16T03:05:11.352ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111082025-12-16T03:05:11.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111092025-12-16T03:05:11.353ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
111102025-12-16T03:05:11.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
111112025-12-16T03:05:11.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
111122025-12-16T03:05:11.354ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111132025-12-16T03:05:11.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111142025-12-16T03:05:11.357ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
111152025-12-16T03:05:11.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
111162025-12-16T03:05:11.357ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
111172025-12-16T03:05:11.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111182025-12-16T03:05:11.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
111192025-12-16T03:05:11.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111202025-12-16T03:05:11.360ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
111212025-12-16T03:05:11.360ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 33.633056ms iteration = 11 status = {"blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","inventory_collection_id":"b6c51ddf-9f27-432f-9d24-9fbc42795d0c","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
111222025-12-16T03:05:11.360ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111232025-12-16T03:05:11.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111242025-12-16T03:05:11.362ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111252025-12-16T03:05:11.365ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
111262025-12-16T03:05:11.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111272025-12-16T03:05:11.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111282025-12-16T03:05:11.367ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111292025-12-16T03:05:11.370ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
111302025-12-16T03:05:11.370ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111312025-12-16T03:05:11.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111322025-12-16T03:05:11.372ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111332025-12-16T03:05:11.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111342025-12-16T03:05:11.373ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111352025-12-16T03:05:11.376ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
111362025-12-16T03:05:11.376ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111372025-12-16T03:05:11.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111382025-12-16T03:05:11.378ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111392025-12-16T03:05:11.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111402025-12-16T03:05:11.380ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111412025-12-16T03:05:11.384ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
111422025-12-16T03:05:11.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111432025-12-16T03:05:11.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111442025-12-16T03:05:11.386ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111452025-12-16T03:05:11.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111462025-12-16T03:05:11.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111472025-12-16T03:05:11.391ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
111482025-12-16T03:05:11.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111492025-12-16T03:05:11.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111502025-12-16T03:05:11.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111512025-12-16T03:05:11.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111522025-12-16T03:05:11.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111532025-12-16T03:05:11.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111542025-12-16T03:05:11.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111552025-12-16T03:05:11.414ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
111562025-12-16T03:05:11.414ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111572025-12-16T03:05:11.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111582025-12-16T03:05:11.416ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111592025-12-16T03:05:11.419ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
111602025-12-16T03:05:11.419ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111612025-12-16T03:05:11.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111622025-12-16T03:05:11.420ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111632025-12-16T03:05:11.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111642025-12-16T03:05:11.422ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111652025-12-16T03:05:11.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111662025-12-16T03:05:11.424ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111672025-12-16T03:05:11.427ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
111682025-12-16T03:05:11.427ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111692025-12-16T03:05:11.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111702025-12-16T03:05:11.427ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 101.01654ms iteration = 22 status = {"physical_disks_added":0}
111712025-12-16T03:05:11.428ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111722025-12-16T03:05:11.431ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
111732025-12-16T03:05:11.431ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111742025-12-16T03:05:11.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111752025-12-16T03:05:11.434ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111762025-12-16T03:05:11.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111772025-12-16T03:05:11.434ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111782025-12-16T03:05:11.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111792025-12-16T03:05:11.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111802025-12-16T03:05:11.451ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
111812025-12-16T03:05:11.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111822025-12-16T03:05:11.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111832025-12-16T03:05:11.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111842025-12-16T03:05:11.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111852025-12-16T03:05:11.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111862025-12-16T03:05:11.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
111872025-12-16T03:05:11.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111882025-12-16T03:05:11.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111892025-12-16T03:05:11.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111902025-12-16T03:05:11.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111912025-12-16T03:05:11.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111922025-12-16T03:05:11.465ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
111932025-12-16T03:05:11.465ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111942025-12-16T03:05:11.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111952025-12-16T03:05:11.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111962025-12-16T03:05:11.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111972025-12-16T03:05:11.468ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111982025-12-16T03:05:11.471ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
111992025-12-16T03:05:11.471ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112002025-12-16T03:05:11.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112012025-12-16T03:05:11.473ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112022025-12-16T03:05:11.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112032025-12-16T03:05:11.475ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112042025-12-16T03:05:11.478ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
112052025-12-16T03:05:11.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112062025-12-16T03:05:11.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112072025-12-16T03:05:11.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112082025-12-16T03:05:11.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112092025-12-16T03:05:11.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112102025-12-16T03:05:11.483ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
112112025-12-16T03:05:11.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112122025-12-16T03:05:11.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112132025-12-16T03:05:11.485ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
112142025-12-16T03:05:11.485ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
112152025-12-16T03:05:11.485ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 9e107f51-1816-4566-aa24-7e27ced41e7c parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
112162025-12-16T03:05:11.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112172025-12-16T03:05:11.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112182025-12-16T03:05:11.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112192025-12-16T03:05:11.489ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
112202025-12-16T03:05:11.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112212025-12-16T03:05:11.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112222025-12-16T03:05:11.492ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
112232025-12-16T03:05:11.492ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
112242025-12-16T03:05:11.492ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 165.442416ms iteration = 18 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
112252025-12-16T03:05:14.239ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 74 reason = Timeout
112262025-12-16T03:05:14.239ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
112272025-12-16T03:05:14.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
112282025-12-16T03:05:14.241ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112292025-12-16T03:05:14.244ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
112302025-12-16T03:05:14.244ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
112312025-12-16T03:05:14.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
112322025-12-16T03:05:14.246ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
112332025-12-16T03:05:14.246ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.607496ms iteration = 74 status = {"config_updated":false}
112342025-12-16T03:05:19.239ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 38 reason = Timeout
112352025-12-16T03:05:19.240ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
112362025-12-16T03:05:19.240ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
112372025-12-16T03:05:19.240ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 75 reason = Timeout
112382025-12-16T03:05:19.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
112392025-12-16T03:05:19.240ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
112402025-12-16T03:05:19.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
112412025-12-16T03:05:19.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
112422025-12-16T03:05:19.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
112432025-12-16T03:05:19.242ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112442025-12-16T03:05:19.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112452025-12-16T03:05:19.242ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
112462025-12-16T03:05:19.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
112472025-12-16T03:05:19.243ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.720108ms iteration = 38 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
112482025-12-16T03:05:19.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
112492025-12-16T03:05:19.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
112502025-12-16T03:05:19.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
112512025-12-16T03:05:19.246ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
112522025-12-16T03:05:19.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
112532025-12-16T03:05:19.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
112542025-12-16T03:05:19.248ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
112552025-12-16T03:05:19.248ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.992581ms iteration = 75 status = {"config_updated":false}
112562025-12-16T03:05:19.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
112572025-12-16T03:05:19.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
112582025-12-16T03:05:19.249ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112592025-12-16T03:05:19.252ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
112602025-12-16T03:05:19.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
112612025-12-16T03:05:19.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
112622025-12-16T03:05:19.265ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
112632025-12-16T03:05:19.265ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.25593ms iteration = 38 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
112642025-12-16T03:05:21.057ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:59290
112652025-12-16T03:05:21.057ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 2844109f-d102-46ff-8c61-a1445f9432d7 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending version = 1.0.0
112662025-12-16T03:05:21.057ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 2844109f-d102-46ff-8c61-a1445f9432d7 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
112672025-12-16T03:05:21.058ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 2844109f-d102-46ff-8c61-a1445f9432d7 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
112682025-12-16T03:05:21.060ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 3176 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 2844109f-d102-46ff-8c61-a1445f9432d7 response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending
112692025-12-16T03:05:21.061ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 2837b2f4-2fdc-4f5e-9d55-cc06088e9ace uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D version = 1.0.0
112702025-12-16T03:05:21.061ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 2837b2f4-2fdc-4f5e-9d55-cc06088e9ace roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
112712025-12-16T03:05:21.061ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 2837b2f4-2fdc-4f5e-9d55-cc06088e9ace resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
112722025-12-16T03:05:21.063ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 2178 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 2837b2f4-2fdc-4f5e-9d55-cc06088e9ace response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D
112732025-12-16T03:05:21.064ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 242 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = defe4d4f-b908-4738-8989-0da2d99e8fe4 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
112742025-12-16T03:05:24.239ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 34 reason = Timeout
112752025-12-16T03:05:24.239ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 26 reason = Timeout
112762025-12-16T03:05:24.239ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
112772025-12-16T03:05:24.239ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
112782025-12-16T03:05:24.239ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 76 reason = Timeout
112792025-12-16T03:05:24.239ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 26 reason = Timeout
112802025-12-16T03:05:24.239ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
112812025-12-16T03:05:24.239ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
112822025-12-16T03:05:24.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
112832025-12-16T03:05:24.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
112842025-12-16T03:05:24.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
112852025-12-16T03:05:24.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
112862025-12-16T03:05:24.241ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
112872025-12-16T03:05:24.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
112882025-12-16T03:05:24.242ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112892025-12-16T03:05:24.242ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112902025-12-16T03:05:24.242ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112912025-12-16T03:05:24.242ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112922025-12-16T03:05:24.245ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
112932025-12-16T03:05:24.245ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
112942025-12-16T03:05:24.245ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
112952025-12-16T03:05:24.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
112962025-12-16T03:05:24.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
112972025-12-16T03:05:24.246ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
112982025-12-16T03:05:24.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
112992025-12-16T03:05:24.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
113002025-12-16T03:05:24.246ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
113012025-12-16T03:05:24.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
113022025-12-16T03:05:24.247ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 7.350223ms iteration = 26 status = "NoSitrep"
113032025-12-16T03:05:24.247ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
113042025-12-16T03:05:24.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
113052025-12-16T03:05:24.248ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
113062025-12-16T03:05:24.248ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
113072025-12-16T03:05:24.248ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.596679ms iteration = 76 status = {"config_updated":false}
113082025-12-16T03:05:24.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
113092025-12-16T03:05:24.249ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = b6c51ddf-9f27-432f-9d24-9fbc42795d0c original_time_started = 2025-12-16 03:05:10.667073 UTC
113102025-12-16T03:05:24.249ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 9.155546ms iteration = 34 status = {"Loaded":{"collection_id":"b6c51ddf-9f27-432f-9d24-9fbc42795d0c","time_loaded":"2025-12-16T03:05:24.239882597Z","time_started":"2025-12-16T03:05:10.667073Z"}}
113112025-12-16T03:05:24.249ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113122025-12-16T03:05:24.251ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
113132025-12-16T03:05:24.251ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
113142025-12-16T03:05:24.251ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 11.976881ms iteration = 26 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
113152025-12-16T03:05:29.239ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 39 reason = Timeout
113162025-12-16T03:05:29.239ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
113172025-12-16T03:05:29.239ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
113182025-12-16T03:05:29.239ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 77 reason = Timeout
113192025-12-16T03:05:29.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113202025-12-16T03:05:29.239ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
113212025-12-16T03:05:29.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
113222025-12-16T03:05:29.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
113232025-12-16T03:05:29.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
113242025-12-16T03:05:29.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113252025-12-16T03:05:29.241ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113262025-12-16T03:05:29.241ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
113272025-12-16T03:05:29.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
113282025-12-16T03:05:29.243ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.017031ms iteration = 39 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
113292025-12-16T03:05:29.245ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
113302025-12-16T03:05:29.245ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
113312025-12-16T03:05:29.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
113322025-12-16T03:05:29.245ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
113332025-12-16T03:05:29.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113342025-12-16T03:05:29.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
113352025-12-16T03:05:29.247ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
113362025-12-16T03:05:29.247ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.8941ms iteration = 77 status = {"config_updated":false}
113372025-12-16T03:05:29.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113382025-12-16T03:05:29.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
113392025-12-16T03:05:29.249ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113402025-12-16T03:05:29.251ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
113412025-12-16T03:05:29.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113422025-12-16T03:05:29.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
113432025-12-16T03:05:29.264ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
113442025-12-16T03:05:29.264ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.082958ms iteration = 39 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
113452025-12-16T03:05:31.065ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 310 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 8c315d89-6d13-41a8-bd77-b71475b4c7e5 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
113462025-12-16T03:05:34.239ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 78 reason = Timeout
113472025-12-16T03:05:34.239ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
113482025-12-16T03:05:34.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
113492025-12-16T03:05:34.241ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113502025-12-16T03:05:34.244ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
113512025-12-16T03:05:34.244ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
113522025-12-16T03:05:34.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
113532025-12-16T03:05:34.246ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
113542025-12-16T03:05:34.246ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.980199ms iteration = 78 status = {"config_updated":false}
113552025-12-16T03:05:39.172ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
113562025-12-16T03:05:39.172ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 14 reason = Timeout
113572025-12-16T03:05:39.172ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
113582025-12-16T03:05:39.172ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
113592025-12-16T03:05:39.172ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
113602025-12-16T03:05:39.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
113612025-12-16T03:05:39.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
113622025-12-16T03:05:39.175ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1346, tv_nsec: 787804571 } })
113632025-12-16T03:05:39.175ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1346, tv_nsec: 787860002 } })
113642025-12-16T03:05:39.175ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
113652025-12-16T03:05:39.176ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1346, tv_nsec: 788397969 } })
113662025-12-16T03:05:39.176ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.601316ms iteration = 14 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
113672025-12-16T03:05:39.225ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
113682025-12-16T03:05:39.226ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 54.037284ms iteration = 14 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
113692025-12-16T03:05:39.227ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
113702025-12-16T03:05:39.227ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
113712025-12-16T03:05:39.228ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c8d81e7-af81-41d6-baf2-2cc4841f9719", "content-length": "26", "date": "Tue, 16 Dec 2025 03:05:39 GMT"} })
113722025-12-16T03:05:39.228ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
113732025-12-16T03:05:39.230ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
113742025-12-16T03:05:39.231ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1346, tv_nsec: 843252514 } })
113752025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 14 reason = Timeout
113762025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 18.581\u{b5}s iteration = 14 status = {"disabled":true,"errors":[],"sps":[]}
113772025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 27 reason = Timeout
113782025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
113792025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 21 reason = Timeout
113802025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
113812025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
113822025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
113832025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 23 reason = Timeout
113842025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 14 reason = Timeout
113852025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
113862025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
113872025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 40 reason = Timeout
113882025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
113892025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
113902025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 21 reason = Timeout
113912025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113922025-12-16T03:05:39.269ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
113932025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
113942025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 14 reason = Timeout
113952025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 35 reason = Timeout
113962025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
113972025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 14 reason = Timeout
113982025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
113992025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
114002025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
114012025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 27 reason = Timeout
114022025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 14 reason = Timeout
114032025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
114042025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
114052025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 79 reason = Timeout
114062025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
114072025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 14 reason = Timeout
114082025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
114092025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
114102025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
114112025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 14 reason = Timeout
114122025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
114132025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 14 reason = Timeout
114142025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
114152025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 14 reason = Timeout
114162025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
114172025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
114182025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
114192025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
114202025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
114212025-12-16T03:05:39.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
114222025-12-16T03:05:39.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
114232025-12-16T03:05:39.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
114242025-12-16T03:05:39.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
114252025-12-16T03:05:39.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
114262025-12-16T03:05:39.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
114272025-12-16T03:05:39.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
114282025-12-16T03:05:39.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
114292025-12-16T03:05:39.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
114302025-12-16T03:05:39.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
114312025-12-16T03:05:39.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
114322025-12-16T03:05:39.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
114332025-12-16T03:05:39.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
114342025-12-16T03:05:39.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
114352025-12-16T03:05:39.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
114362025-12-16T03:05:39.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
114372025-12-16T03:05:39.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
114382025-12-16T03:05:39.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
114392025-12-16T03:05:39.270ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
114402025-12-16T03:05:39.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
114412025-12-16T03:05:39.272ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114422025-12-16T03:05:39.272ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114432025-12-16T03:05:39.272ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114442025-12-16T03:05:39.272ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114452025-12-16T03:05:39.272ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114462025-12-16T03:05:39.272ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114472025-12-16T03:05:39.272ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 3.274831ms iteration = 14 status = {"errors":[],"finish_invoked_ok":[]}
114482025-12-16T03:05:39.272ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114492025-12-16T03:05:39.272ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 100.052277ms iteration = 14 status = {}
114502025-12-16T03:05:39.272ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114512025-12-16T03:05:39.273ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
114522025-12-16T03:05:39.273ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
114532025-12-16T03:05:39.273ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
114542025-12-16T03:05:39.273ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
114552025-12-16T03:05:39.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
114562025-12-16T03:05:39.274ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114572025-12-16T03:05:39.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
114582025-12-16T03:05:39.275ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114592025-12-16T03:05:39.276ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
114602025-12-16T03:05:39.276ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
114612025-12-16T03:05:39.276ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
114622025-12-16T03:05:39.277ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
114632025-12-16T03:05:39.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
114642025-12-16T03:05:39.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
114652025-12-16T03:05:39.278ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
114662025-12-16T03:05:39.278ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
114672025-12-16T03:05:39.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
114682025-12-16T03:05:39.279ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
114692025-12-16T03:05:39.279ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
114702025-12-16T03:05:39.279ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
114712025-12-16T03:05:39.279ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
114722025-12-16T03:05:39.279ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
114732025-12-16T03:05:39.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
114742025-12-16T03:05:39.279ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
114752025-12-16T03:05:39.279ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
114762025-12-16T03:05:39.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
114772025-12-16T03:05:39.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
114782025-12-16T03:05:39.280ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
114792025-12-16T03:05:39.280ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114802025-12-16T03:05:39.280ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
114812025-12-16T03:05:39.280ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
114822025-12-16T03:05:39.280ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
114832025-12-16T03:05:39.280ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
114842025-12-16T03:05:39.280ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
114852025-12-16T03:05:39.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
114862025-12-16T03:05:39.281ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 11.766339ms iteration = 27 status = "NoSitrep"
114872025-12-16T03:05:39.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
114882025-12-16T03:05:39.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
114892025-12-16T03:05:39.281ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
114902025-12-16T03:05:39.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
114912025-12-16T03:05:39.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
114922025-12-16T03:05:39.281ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
114932025-12-16T03:05:39.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
114942025-12-16T03:05:39.282ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
114952025-12-16T03:05:39.283ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1346, tv_nsec: 895236433 } })
114962025-12-16T03:05:39.283ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
114972025-12-16T03:05:39.283ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
114982025-12-16T03:05:39.283ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 13.506812ms iteration = 79 status = {"config_updated":false}
114992025-12-16T03:05:39.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
115002025-12-16T03:05:39.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
115012025-12-16T03:05:39.283ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
115022025-12-16T03:05:39.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
115032025-12-16T03:05:39.283ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
115042025-12-16T03:05:39.283ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = b6c51ddf-9f27-432f-9d24-9fbc42795d0c original_time_started = 2025-12-16 03:05:10.667073 UTC
115052025-12-16T03:05:39.283ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 14.391533ms iteration = 35 status = {"Loaded":{"collection_id":"b6c51ddf-9f27-432f-9d24-9fbc42795d0c","time_loaded":"2025-12-16T03:05:39.269467839Z","time_started":"2025-12-16T03:05:10.667073Z"}}
115062025-12-16T03:05:39.283ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
115072025-12-16T03:05:39.283ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1346, tv_nsec: 896159504 } })
115082025-12-16T03:05:39.284ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
115092025-12-16T03:05:39.284ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
115102025-12-16T03:05:39.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
115112025-12-16T03:05:39.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
115122025-12-16T03:05:39.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
115132025-12-16T03:05:39.284ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
115142025-12-16T03:05:39.284ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115152025-12-16T03:05:39.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
115162025-12-16T03:05:39.284ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115172025-12-16T03:05:39.285ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115182025-12-16T03:05:39.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
115192025-12-16T03:05:39.287ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
115202025-12-16T03:05:39.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
115212025-12-16T03:05:39.288ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
115222025-12-16T03:05:39.288ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
115232025-12-16T03:05:39.288ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
115242025-12-16T03:05:39.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
115252025-12-16T03:05:39.289ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
115262025-12-16T03:05:39.289ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 19.595898ms iteration = 27 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
115272025-12-16T03:05:39.293ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 24.316858ms iteration = 14 status = 0
115282025-12-16T03:05:39.296ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
115292025-12-16T03:05:39.296ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 27.517718ms iteration = 21 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
115302025-12-16T03:05:39.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
115312025-12-16T03:05:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
115322025-12-16T03:05:39.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115332025-12-16T03:05:39.319ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 49.841302ms iteration = 14 status = {"errors":[],"garbage_collect_requested":[]}
115342025-12-16T03:05:39.319ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
115352025-12-16T03:05:39.319ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
115362025-12-16T03:05:39.320ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 50.710173ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
115372025-12-16T03:05:39.320ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "03b13c88-522a-4059-907a-d8e6662c98ad", "content-length": "26", "date": "Tue, 16 Dec 2025 03:05:39 GMT"} })
115382025-12-16T03:05:39.320ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
115392025-12-16T03:05:39.323ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
115402025-12-16T03:05:39.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
115412025-12-16T03:05:39.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
115422025-12-16T03:05:39.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
115432025-12-16T03:05:39.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
115442025-12-16T03:05:39.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
115452025-12-16T03:05:39.328ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
115462025-12-16T03:05:39.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
115472025-12-16T03:05:39.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115482025-12-16T03:05:39.330ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
115492025-12-16T03:05:39.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
115502025-12-16T03:05:39.331ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 61.943745ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
115512025-12-16T03:05:39.333ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
115522025-12-16T03:05:39.333ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
115532025-12-16T03:05:39.333ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 64.223203ms iteration = 14 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
115542025-12-16T03:05:39.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
115552025-12-16T03:05:39.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
115562025-12-16T03:05:39.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
115572025-12-16T03:05:39.343ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 73.441591ms iteration = 14 status = {"errors":[],"requests_created_ok":[]}
115582025-12-16T03:05:39.344ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
115592025-12-16T03:05:39.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
115602025-12-16T03:05:39.346ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 76.902634ms iteration = 40 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
115612025-12-16T03:05:39.354ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
115622025-12-16T03:05:39.354ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 85.549114ms iteration = 40 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
115632025-12-16T03:05:39.355ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
115642025-12-16T03:05:39.355ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 85.638715ms iteration = 14 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
115652025-12-16T03:05:39.359ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 90.108792ms iteration = 23 status = {"physical_disks_added":0}
115662025-12-16T03:05:39.362ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1346, tv_nsec: 974341185 } })
115672025-12-16T03:05:39.377ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 107.893977ms iteration = 14 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
115682025-12-16T03:05:39.413ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1347, tv_nsec: 25696446 } })
115692025-12-16T03:05:39.433ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
115702025-12-16T03:05:39.433ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
115712025-12-16T03:05:39.435ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
115722025-12-16T03:05:39.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
115732025-12-16T03:05:39.437ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "80e43184-83ca-48cd-b072-9d5e443afda5", "content-length": "372", "date": "Tue, 16 Dec 2025 03:05:39 GMT"} })
115742025-12-16T03:05:39.437ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
115752025-12-16T03:05:39.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
115762025-12-16T03:05:39.438ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115772025-12-16T03:05:39.440ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
115782025-12-16T03:05:39.440ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
115792025-12-16T03:05:39.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
115802025-12-16T03:05:39.442ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115812025-12-16T03:05:39.444ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
115822025-12-16T03:05:39.444ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
115832025-12-16T03:05:39.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
115842025-12-16T03:05:39.450ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
115852025-12-16T03:05:39.450ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
115862025-12-16T03:05:39.452ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f018df7d-054a-4cd7-83f6-4c675870c39d", "date": "Tue, 16 Dec 2025 03:05:39 GMT"} })
115872025-12-16T03:05:39.452ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
115882025-12-16T03:05:39.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
115892025-12-16T03:05:39.455ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
115902025-12-16T03:05:39.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
115912025-12-16T03:05:39.457ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
115922025-12-16T03:05:39.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
115932025-12-16T03:05:39.460ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
115942025-12-16T03:05:39.460ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
115952025-12-16T03:05:39.461ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115962025-12-16T03:05:39.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115972025-12-16T03:05:39.463ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
115982025-12-16T03:05:39.463ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115992025-12-16T03:05:39.464ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "01df5e0d-06a0-4f5d-a8a4-1c14a4266ab9", "date": "Tue, 16 Dec 2025 03:05:39 GMT"} })
116002025-12-16T03:05:39.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
116012025-12-16T03:05:39.465ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 196.449168ms iteration = 21 status = {}
116022025-12-16T03:05:39.476ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
116032025-12-16T03:05:39.476ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
116042025-12-16T03:05:39.477ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
116052025-12-16T03:05:39.477ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
116062025-12-16T03:05:39.477ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
116072025-12-16T03:05:39.477ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
116082025-12-16T03:05:39.477ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
116092025-12-16T03:05:39.477ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
116102025-12-16T03:05:39.477ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
116112025-12-16T03:05:39.477ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
116122025-12-16T03:05:39.478ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
116132025-12-16T03:05:39.478ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
116142025-12-16T03:05:39.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
116152025-12-16T03:05:39.481ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
116162025-12-16T03:05:39.481ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
116172025-12-16T03:05:39.485ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
116182025-12-16T03:05:39.490ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "000c603d-a2c0-4c2d-a140-86dddd2d5891", "content-length": "691", "date": "Tue, 16 Dec 2025 03:05:39 GMT"} })
116192025-12-16T03:05:39.490ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
116202025-12-16T03:05:39.490ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
116212025-12-16T03:05:39.490ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
116222025-12-16T03:05:39.491ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 221.922111ms iteration = 14 status = {}
116232025-12-16T03:05:39.648ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "646eeb76-c945-43ef-ab47-fadab7696e44", "date": "Tue, 16 Dec 2025 03:05:39 GMT"} })
116242025-12-16T03:05:39.648ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
116252025-12-16T03:05:39.648ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
116262025-12-16T03:05:39.648ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f69b159-28fe-46ab-b310-465d8ea76b3d", "content-length": "682", "date": "Tue, 16 Dec 2025 03:05:39 GMT"} })
116272025-12-16T03:05:39.648ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
116282025-12-16T03:05:39.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
116292025-12-16T03:05:39.652ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
116302025-12-16T03:05:39.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
116312025-12-16T03:05:39.655ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
116322025-12-16T03:05:39.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
116332025-12-16T03:05:39.657ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
116342025-12-16T03:05:39.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
116352025-12-16T03:05:39.663ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 394.496608ms iteration = 14 status = {}
116362025-12-16T03:05:41.065ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 296 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 5d6f95c7-77ab-40da-bb89-ca3bf5a1c8f9 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
116372025-12-16T03:05:44.270ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 80 reason = Timeout
116382025-12-16T03:05:44.270ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
116392025-12-16T03:05:44.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
116402025-12-16T03:05:44.273ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116412025-12-16T03:05:44.276ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
116422025-12-16T03:05:44.276ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
116432025-12-16T03:05:44.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
116442025-12-16T03:05:44.279ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
116452025-12-16T03:05:44.279ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.606649ms iteration = 80 status = {"config_updated":false}
116462025-12-16T03:05:49.271ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 41 reason = Timeout
116472025-12-16T03:05:49.271ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
116482025-12-16T03:05:49.271ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
116492025-12-16T03:05:49.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116502025-12-16T03:05:49.271ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 81 reason = Timeout
116512025-12-16T03:05:49.271ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
116522025-12-16T03:05:49.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
116532025-12-16T03:05:49.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
116542025-12-16T03:05:49.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
116552025-12-16T03:05:49.273ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
116562025-12-16T03:05:49.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116572025-12-16T03:05:49.273ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116582025-12-16T03:05:49.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
116592025-12-16T03:05:49.275ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.677596ms iteration = 41 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
116602025-12-16T03:05:49.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
116612025-12-16T03:05:49.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
116622025-12-16T03:05:49.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
116632025-12-16T03:05:49.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
116642025-12-16T03:05:49.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116652025-12-16T03:05:49.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
116662025-12-16T03:05:49.279ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
116672025-12-16T03:05:49.279ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.757158ms iteration = 81 status = {"config_updated":false}
116682025-12-16T03:05:49.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116692025-12-16T03:05:49.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
116702025-12-16T03:05:49.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116712025-12-16T03:05:49.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
116722025-12-16T03:05:49.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116732025-12-16T03:05:49.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
116742025-12-16T03:05:49.299ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
116752025-12-16T03:05:49.299ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.335299ms iteration = 41 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
116762025-12-16T03:05:51.058ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 6c5593c3-4f36-4724-ac27-1642dbe859d1 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending version = 1.0.0
116772025-12-16T03:05:51.058ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 6c5593c3-4f36-4724-ac27-1642dbe859d1 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
116782025-12-16T03:05:51.058ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 6c5593c3-4f36-4724-ac27-1642dbe859d1 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
116792025-12-16T03:05:51.061ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 3698 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 6c5593c3-4f36-4724-ac27-1642dbe859d1 response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending
116802025-12-16T03:05:51.062ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 9a2e2007-9cb3-41f4-9373-8d014d8a3642 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D version = 1.0.0
116812025-12-16T03:05:51.062ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 9a2e2007-9cb3-41f4-9373-8d014d8a3642 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
116822025-12-16T03:05:51.062ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 9a2e2007-9cb3-41f4-9373-8d014d8a3642 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
116832025-12-16T03:05:51.064ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 2195 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 9a2e2007-9cb3-41f4-9373-8d014d8a3642 response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D
116842025-12-16T03:05:51.065ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 251 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 0885af5a-ccee-4697-ba7f-797d757169c0 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
116852025-12-16T03:05:54.271ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 28 reason = Timeout
116862025-12-16T03:05:54.271ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 28 reason = Timeout
116872025-12-16T03:05:54.271ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
116882025-12-16T03:05:54.271ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
116892025-12-16T03:05:54.271ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 82 reason = Timeout
116902025-12-16T03:05:54.271ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
116912025-12-16T03:05:54.271ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 36 reason = Timeout
116922025-12-16T03:05:54.271ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
116932025-12-16T03:05:54.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
116942025-12-16T03:05:54.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
116952025-12-16T03:05:54.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
116962025-12-16T03:05:54.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
116972025-12-16T03:05:54.272ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
116982025-12-16T03:05:54.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
116992025-12-16T03:05:54.273ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117002025-12-16T03:05:54.273ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117012025-12-16T03:05:54.273ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117022025-12-16T03:05:54.274ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117032025-12-16T03:05:54.276ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
117042025-12-16T03:05:54.277ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
117052025-12-16T03:05:54.277ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
117062025-12-16T03:05:54.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
117072025-12-16T03:05:54.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
117082025-12-16T03:05:54.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
117092025-12-16T03:05:54.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
117102025-12-16T03:05:54.277ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
117112025-12-16T03:05:54.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
117122025-12-16T03:05:54.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
117132025-12-16T03:05:54.278ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 7.821559ms iteration = 28 status = "NoSitrep"
117142025-12-16T03:05:54.279ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
117152025-12-16T03:05:54.279ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.70718ms iteration = 82 status = {"config_updated":false}
117162025-12-16T03:05:54.279ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
117172025-12-16T03:05:54.280ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = b6c51ddf-9f27-432f-9d24-9fbc42795d0c original_time_started = 2025-12-16 03:05:10.667073 UTC
117182025-12-16T03:05:54.280ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.884962ms iteration = 36 status = {"Loaded":{"collection_id":"b6c51ddf-9f27-432f-9d24-9fbc42795d0c","time_loaded":"2025-12-16T03:05:54.271206858Z","time_started":"2025-12-16T03:05:10.667073Z"}}
117192025-12-16T03:05:54.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
117202025-12-16T03:05:54.280ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
117212025-12-16T03:05:54.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
117222025-12-16T03:05:54.282ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117232025-12-16T03:05:54.283ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
117242025-12-16T03:05:54.283ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
117252025-12-16T03:05:54.283ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 12.744391ms iteration = 28 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
117262025-12-16T03:05:59.271ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 42 reason = Timeout
117272025-12-16T03:05:59.271ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
117282025-12-16T03:05:59.271ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
117292025-12-16T03:05:59.271ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 83 reason = Timeout
117302025-12-16T03:05:59.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
117312025-12-16T03:05:59.271ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
117322025-12-16T03:05:59.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
117332025-12-16T03:05:59.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
117342025-12-16T03:05:59.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
117352025-12-16T03:05:59.273ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117362025-12-16T03:05:59.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117372025-12-16T03:05:59.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
117382025-12-16T03:05:59.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
117392025-12-16T03:05:59.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
117402025-12-16T03:05:59.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
117412025-12-16T03:05:59.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
117422025-12-16T03:05:59.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
117432025-12-16T03:05:59.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
117442025-12-16T03:05:59.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
117452025-12-16T03:05:59.279ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
117462025-12-16T03:05:59.279ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.95295ms iteration = 83 status = {"config_updated":false}
117472025-12-16T03:05:59.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117482025-12-16T03:05:59.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
117492025-12-16T03:05:59.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
117502025-12-16T03:05:59.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
117512025-12-16T03:05:59.295ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
117522025-12-16T03:05:59.295ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.191827ms iteration = 42 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
117532025-12-16T03:05:59.309ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
117542025-12-16T03:05:59.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
117552025-12-16T03:05:59.357ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 86.679128ms iteration = 42 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
117562025-12-16T03:06:01.065ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 271 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 46ef44a3-2c15-4f60-a682-3078d9fdbcac response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
117572025-12-16T03:06:04.270ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 84 reason = Timeout
117582025-12-16T03:06:04.270ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
117592025-12-16T03:06:04.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
117602025-12-16T03:06:04.273ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117612025-12-16T03:06:04.276ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
117622025-12-16T03:06:04.276ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
117632025-12-16T03:06:04.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
117642025-12-16T03:06:04.298ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
117652025-12-16T03:06:04.298ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 27.951584ms iteration = 84 status = {"config_updated":false}
117662025-12-16T03:06:06.058ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 1a25bc6c-a992-4e04-ba9f-e9e75b1ef717 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending version = 1.0.0
117672025-12-16T03:06:06.058ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 1a25bc6c-a992-4e04-ba9f-e9e75b1ef717 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
117682025-12-16T03:06:06.059ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 1a25bc6c-a992-4e04-ba9f-e9e75b1ef717 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
117692025-12-16T03:06:06.061ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 3119 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 1a25bc6c-a992-4e04-ba9f-e9e75b1ef717 response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending
117702025-12-16T03:06:06.062ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = c7abcb76-e803-4cee-bff2-30885115b83f uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D version = 1.0.0
117712025-12-16T03:06:06.062ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = c7abcb76-e803-4cee-bff2-30885115b83f roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
117722025-12-16T03:06:06.062ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = c7abcb76-e803-4cee-bff2-30885115b83f resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
117732025-12-16T03:06:06.064ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 2183 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = c7abcb76-e803-4cee-bff2-30885115b83f response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D
117742025-12-16T03:06:09.173ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
117752025-12-16T03:06:09.173ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
117762025-12-16T03:06:09.173ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
117772025-12-16T03:06:09.173ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
117782025-12-16T03:06:09.173ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 15 reason = Timeout
117792025-12-16T03:06:09.173ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
117802025-12-16T03:06:09.173ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
117812025-12-16T03:06:09.173ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
117822025-12-16T03:06:09.173ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
117832025-12-16T03:06:09.173ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
117842025-12-16T03:06:09.173ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
117852025-12-16T03:06:09.173ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
117862025-12-16T03:06:09.173ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
117872025-12-16T03:06:09.173ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
117882025-12-16T03:06:09.173ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
117892025-12-16T03:06:09.173ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
117902025-12-16T03:06:09.173ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
117912025-12-16T03:06:09.173ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
117922025-12-16T03:06:09.173ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
117932025-12-16T03:06:09.173ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 01024e9a-d5fa-4122-8959-6a135f273dcd step component = ExternalNetworking step id = Ensure
117942025-12-16T03:06:09.173ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
117952025-12-16T03:06:09.173ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
117962025-12-16T03:06:09.173ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-12-16 02:56:09.173705010 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
117972025-12-16T03:06:09.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
117982025-12-16T03:06:09.173ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
117992025-12-16T03:06:09.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
118002025-12-16T03:06:09.174ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1a45cd23-3d0a-43eb-adce-62c7b712baf1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1376, tv_nsec: 786007119 } })
118012025-12-16T03:06:09.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
118022025-12-16T03:06:09.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
118032025-12-16T03:06:09.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
118042025-12-16T03:06:09.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
118052025-12-16T03:06:09.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
118062025-12-16T03:06:09.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
118072025-12-16T03:06:09.174ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("49a51a59-9919-48ae-8182-4070a8430160.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6af60386-8771-4401-9d4d-e93774e519f0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 1376, tv_nsec: 786156990 } })
118082025-12-16T03:06:09.174ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
118092025-12-16T03:06:09.174ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.256596ms iteration = 8 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
118102025-12-16T03:06:09.174ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
118112025-12-16T03:06:09.175ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.294216ms iteration = 8 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
118122025-12-16T03:06:09.175ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
118132025-12-16T03:06:09.175ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
118142025-12-16T03:06:09.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
118152025-12-16T03:06:09.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
118162025-12-16T03:06:09.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
118172025-12-16T03:06:09.176ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118182025-12-16T03:06:09.176ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-12-16 02:56:09.173705010 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
118192025-12-16T03:06:09.176ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.1423ms iteration = 8 status = {"expiration":"2025-12-16T02:56:09.173705010Z","pruned":[]}
118202025-12-16T03:06:09.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
118212025-12-16T03:06:09.177ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1376, tv_nsec: 789016807 } })
118222025-12-16T03:06:09.177ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1376, tv_nsec: 789247730 } })
118232025-12-16T03:06:09.177ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118242025-12-16T03:06:09.177ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118252025-12-16T03:06:09.178ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
118262025-12-16T03:06:09.178ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1376, tv_nsec: 790234582 } })
118272025-12-16T03:06:09.179ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.996466ms iteration = 15 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
118282025-12-16T03:06:09.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
118292025-12-16T03:06:09.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
118302025-12-16T03:06:09.180ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
118312025-12-16T03:06:09.180ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
118322025-12-16T03:06:09.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
118332025-12-16T03:06:09.182ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
118342025-12-16T03:06:09.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
118352025-12-16T03:06:09.184ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
118362025-12-16T03:06:09.184ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
118372025-12-16T03:06:09.185ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118382025-12-16T03:06:09.187ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
118392025-12-16T03:06:09.187ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
118402025-12-16T03:06:09.187ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
118412025-12-16T03:06:09.187ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
118422025-12-16T03:06:09.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
118432025-12-16T03:06:09.189ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118442025-12-16T03:06:09.189ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118452025-12-16T03:06:09.192ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
118462025-12-16T03:06:09.192ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
118472025-12-16T03:06:09.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
118482025-12-16T03:06:09.195ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
118492025-12-16T03:06:09.195ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
118502025-12-16T03:06:09.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
118512025-12-16T03:06:09.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
118522025-12-16T03:06:09.196ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
118532025-12-16T03:06:09.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
118542025-12-16T03:06:09.197ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118552025-12-16T03:06:09.198ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
118562025-12-16T03:06:09.200ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
118572025-12-16T03:06:09.202ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
118582025-12-16T03:06:09.202ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
118592025-12-16T03:06:09.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
118602025-12-16T03:06:09.204ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
118612025-12-16T03:06:09.204ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
118622025-12-16T03:06:09.204ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
118632025-12-16T03:06:09.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
118642025-12-16T03:06:09.205ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
118652025-12-16T03:06:09.207ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118662025-12-16T03:06:09.208ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
118672025-12-16T03:06:09.210ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
118682025-12-16T03:06:09.212ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
118692025-12-16T03:06:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
118702025-12-16T03:06:09.212ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
118712025-12-16T03:06:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
118722025-12-16T03:06:09.213ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-12-16T02:59:05.020237Z, time_modified: 2025-12-16T02:59:05.020237Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }) }
118732025-12-16T03:06:09.213ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 39.47179ms iteration = 8 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]}},"default_endpoint":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
118742025-12-16T03:06:09.214ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 19 reason = Timeout
118752025-12-16T03:06:09.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
118762025-12-16T03:06:09.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
118772025-12-16T03:06:09.215ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 8 reason = Timeout
118782025-12-16T03:06:09.215ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 8 reason = Timeout
118792025-12-16T03:06:09.215ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
118802025-12-16T03:06:09.215ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
118812025-12-16T03:06:09.215ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 8 reason = Timeout
118822025-12-16T03:06:09.215ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
118832025-12-16T03:06:09.215ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
118842025-12-16T03:06:09.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
118852025-12-16T03:06:09.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
118862025-12-16T03:06:09.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
118872025-12-16T03:06:09.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118882025-12-16T03:06:09.220ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
118892025-12-16T03:06:09.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
118902025-12-16T03:06:09.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
118912025-12-16T03:06:09.221ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
118922025-12-16T03:06:09.221ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
118932025-12-16T03:06:09.221ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-12-16 02:59:16.791641 UTC dns_group = internal generation = 1
118942025-12-16T03:06:09.221ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 47.739085ms iteration = 8 status = {"generation":1}
118952025-12-16T03:06:09.224ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
118962025-12-16T03:06:09.228ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118972025-12-16T03:06:09.229ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
118982025-12-16T03:06:09.229ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118992025-12-16T03:06:09.231ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 25 reason = Timeout
119002025-12-16T03:06:09.231ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
119012025-12-16T03:06:09.231ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
119022025-12-16T03:06:09.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119032025-12-16T03:06:09.231ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 23.781\u{b5}s iteration = 25 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
119042025-12-16T03:06:09.231ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 8 reason = Timeout
119052025-12-16T03:06:09.232ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 8 reason = Timeout
119062025-12-16T03:06:09.232ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 8 reason = Timeout
119072025-12-16T03:06:09.232ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
119082025-12-16T03:06:09.232ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
119092025-12-16T03:06:09.232ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
119102025-12-16T03:06:09.232ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
119112025-12-16T03:06:09.232ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 8 reason = Timeout
119122025-12-16T03:06:09.232ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
119132025-12-16T03:06:09.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119142025-12-16T03:06:09.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
119152025-12-16T03:06:09.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
119162025-12-16T03:06:09.232ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
119172025-12-16T03:06:09.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
119182025-12-16T03:06:09.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
119192025-12-16T03:06:09.234ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
119202025-12-16T03:06:09.234ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
119212025-12-16T03:06:09.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
119222025-12-16T03:06:09.235ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 3.382822ms iteration = 8 status = {"errors":[],"found_port_ok":[]}
119232025-12-16T03:06:09.235ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
119242025-12-16T03:06:09.235ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 3.359483ms iteration = 8 status = {"by_rx":{},"error":null}
119252025-12-16T03:06:09.236ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119262025-12-16T03:06:09.236ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119272025-12-16T03:06:09.236ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
119282025-12-16T03:06:09.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119292025-12-16T03:06:09.238ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
119302025-12-16T03:06:09.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119312025-12-16T03:06:09.239ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
119322025-12-16T03:06:09.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
119332025-12-16T03:06:09.240ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
119342025-12-16T03:06:09.240ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
119352025-12-16T03:06:09.240ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
119362025-12-16T03:06:09.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
119372025-12-16T03:06:09.241ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 01024e9a-d5fa-4122-8959-6a135f273dcd step component = SledList step id = Fetch
119382025-12-16T03:06:09.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
119392025-12-16T03:06:09.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
119402025-12-16T03:06:09.242ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
119412025-12-16T03:06:09.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119422025-12-16T03:06:09.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119432025-12-16T03:06:09.242ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 11.03625ms iteration = 8 status = {"dispatched":[],"errors":[],"glob_version":"212.0.0","globs_reprocessed":{},"no_receivers":[]}
119442025-12-16T03:06:09.270ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 15 reason = Timeout
119452025-12-16T03:06:09.270ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 28.701\u{b5}s iteration = 15 status = {"disabled":true,"errors":[],"sps":[]}
119462025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 24 reason = Timeout
119472025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 37 reason = Timeout
119482025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 29 reason = Timeout
119492025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
119502025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
119512025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
119522025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 15 reason = Timeout
119532025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
119542025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
119552025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
119562025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 15 reason = Timeout
119572025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
119582025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
119592025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 29 reason = Timeout
119602025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
119612025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
119622025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 22 reason = Timeout
119632025-12-16T03:06:09.271ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
119642025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
119652025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
119662025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 43 reason = Timeout
119672025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
119682025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 15 reason = Timeout
119692025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
119702025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 85 reason = Timeout
119712025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
119722025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 15 reason = Timeout
119732025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
119742025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 15 reason = Timeout
119752025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
119762025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
119772025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
119782025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
119792025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 22 reason = Timeout
119802025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
119812025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
119822025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 15 reason = Timeout
119832025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
119842025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
119852025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
119862025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 15 reason = Timeout
119872025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
119882025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
119892025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
119902025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
119912025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
119922025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
119932025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
119942025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
119952025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
119962025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
119972025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
119982025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
119992025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
120002025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
120012025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
120022025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
120032025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
120042025-12-16T03:06:09.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
120052025-12-16T03:06:09.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
120062025-12-16T03:06:09.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
120072025-12-16T03:06:09.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
120082025-12-16T03:06:09.275ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120092025-12-16T03:06:09.276ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120102025-12-16T03:06:09.279ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120112025-12-16T03:06:09.280ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
120122025-12-16T03:06:09.280ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
120132025-12-16T03:06:09.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
120142025-12-16T03:06:09.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120152025-12-16T03:06:09.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
120162025-12-16T03:06:09.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
120172025-12-16T03:06:09.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
120182025-12-16T03:06:09.284ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
120192025-12-16T03:06:09.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
120202025-12-16T03:06:09.285ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
120212025-12-16T03:06:09.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
120222025-12-16T03:06:09.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
120232025-12-16T03:06:09.285ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
120242025-12-16T03:06:09.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
120252025-12-16T03:06:09.286ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120262025-12-16T03:06:09.287ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
120272025-12-16T03:06:09.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
120282025-12-16T03:06:09.289ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
120292025-12-16T03:06:09.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120302025-12-16T03:06:09.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
120312025-12-16T03:06:09.290ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
120322025-12-16T03:06:09.290ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
120332025-12-16T03:06:09.290ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
120342025-12-16T03:06:09.290ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
120352025-12-16T03:06:09.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
120362025-12-16T03:06:09.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
120372025-12-16T03:06:09.290ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
120382025-12-16T03:06:09.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
120392025-12-16T03:06:09.294ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
120402025-12-16T03:06:09.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
120412025-12-16T03:06:09.297ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
120422025-12-16T03:06:09.297ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 26.644667ms iteration = 85 status = {"config_updated":false}
120432025-12-16T03:06:09.312ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
120442025-12-16T03:06:09.312ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
120452025-12-16T03:06:09.312ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 96.920328ms iteration = 8 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
120462025-12-16T03:06:09.314ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120472025-12-16T03:06:09.318ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = b6c51ddf-9f27-432f-9d24-9fbc42795d0c original_time_started = 2025-12-16 03:05:10.667073 UTC
120482025-12-16T03:06:09.318ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 47.871726ms iteration = 37 status = {"Loaded":{"collection_id":"b6c51ddf-9f27-432f-9d24-9fbc42795d0c","time_loaded":"2025-12-16T03:06:09.270938072Z","time_started":"2025-12-16T03:05:10.667073Z"}}
120492025-12-16T03:06:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
120502025-12-16T03:06:09.320ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
120512025-12-16T03:06:09.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
120522025-12-16T03:06:09.322ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
120532025-12-16T03:06:09.322ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
120542025-12-16T03:06:09.322ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 106.835794ms iteration = 8 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
120552025-12-16T03:06:09.322ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120562025-12-16T03:06:09.326ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
120572025-12-16T03:06:09.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120582025-12-16T03:06:09.328ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120592025-12-16T03:06:09.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120602025-12-16T03:06:09.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
120612025-12-16T03:06:09.330ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
120622025-12-16T03:06:09.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
120632025-12-16T03:06:09.332ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
120642025-12-16T03:06:09.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
120652025-12-16T03:06:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
120662025-12-16T03:06:09.333ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 62.290649ms iteration = 15 status = 0
120672025-12-16T03:06:09.333ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
120682025-12-16T03:06:09.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120692025-12-16T03:06:09.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
120702025-12-16T03:06:09.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
120712025-12-16T03:06:09.334ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
120722025-12-16T03:06:09.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
120732025-12-16T03:06:09.336ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
120742025-12-16T03:06:09.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
120752025-12-16T03:06:09.336ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120762025-12-16T03:06:09.338ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
120772025-12-16T03:06:09.338ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
120782025-12-16T03:06:09.338ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120792025-12-16T03:06:09.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
120802025-12-16T03:06:09.342ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
120812025-12-16T03:06:09.342ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120822025-12-16T03:06:09.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
120832025-12-16T03:06:09.345ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
120842025-12-16T03:06:09.345ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
120852025-12-16T03:06:09.345ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
120862025-12-16T03:06:09.345ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
120872025-12-16T03:06:09.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
120882025-12-16T03:06:09.345ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a150336-c57e-466b-bd55-96677856f499", "content-length": "26", "date": "Tue, 16 Dec 2025 03:06:09 GMT"} })
120892025-12-16T03:06:09.345ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
120902025-12-16T03:06:09.348ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
120912025-12-16T03:06:09.349ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1376, tv_nsec: 961342070 } })
120922025-12-16T03:06:09.352ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120932025-12-16T03:06:09.353ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 82.09189ms iteration = 15 status = {"errors":[],"garbage_collect_requested":[]}
120942025-12-16T03:06:09.353ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120952025-12-16T03:06:09.354ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120962025-12-16T03:06:09.356ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
120972025-12-16T03:06:09.356ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
120982025-12-16T03:06:09.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
120992025-12-16T03:06:09.357ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 86.488576ms iteration = 15 status = {"errors":[],"finish_invoked_ok":[]}
121002025-12-16T03:06:09.358ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
121012025-12-16T03:06:09.359ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
121022025-12-16T03:06:09.359ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
121032025-12-16T03:06:09.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
121042025-12-16T03:06:09.362ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
121052025-12-16T03:06:09.362ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
121062025-12-16T03:06:09.362ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 91.53301ms iteration = 29 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
121072025-12-16T03:06:09.362ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
121082025-12-16T03:06:09.362ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121092025-12-16T03:06:09.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
121102025-12-16T03:06:09.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
121112025-12-16T03:06:09.363ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121122025-12-16T03:06:09.365ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 94.634279ms iteration = 29 status = "NoSitrep"
121132025-12-16T03:06:09.366ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121142025-12-16T03:06:09.367ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
121152025-12-16T03:06:09.367ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
121162025-12-16T03:06:09.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
121172025-12-16T03:06:09.368ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
121182025-12-16T03:06:09.368ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
121192025-12-16T03:06:09.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
121202025-12-16T03:06:09.369ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
121212025-12-16T03:06:09.369ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121222025-12-16T03:06:09.370ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 99.051285ms iteration = 15 status = {"errors":[],"requests_created_ok":[]}
121232025-12-16T03:06:09.370ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
121242025-12-16T03:06:09.370ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
121252025-12-16T03:06:09.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
121262025-12-16T03:06:09.372ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1376, tv_nsec: 984165119 } })
121272025-12-16T03:06:09.373ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
121282025-12-16T03:06:09.373ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
121292025-12-16T03:06:09.373ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
121302025-12-16T03:06:09.373ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1376, tv_nsec: 985138452 } })
121312025-12-16T03:06:09.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
121322025-12-16T03:06:09.373ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
121332025-12-16T03:06:09.373ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
121342025-12-16T03:06:09.373ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
121352025-12-16T03:06:09.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
121362025-12-16T03:06:09.380ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 109.904483ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
121372025-12-16T03:06:09.387ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
121382025-12-16T03:06:09.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
121392025-12-16T03:06:09.390ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121402025-12-16T03:06:09.393ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
121412025-12-16T03:06:09.393ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 122.793005ms iteration = 43 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
121422025-12-16T03:06:09.394ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
121432025-12-16T03:06:09.394ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
121442025-12-16T03:06:09.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
121452025-12-16T03:06:09.398ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
121462025-12-16T03:06:09.398ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 127.279403ms iteration = 22 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
121472025-12-16T03:06:09.406ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
121482025-12-16T03:06:09.411ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 140.563921ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
121492025-12-16T03:06:09.418ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
121502025-12-16T03:06:09.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
121512025-12-16T03:06:09.423ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
121522025-12-16T03:06:09.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
121532025-12-16T03:06:09.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121542025-12-16T03:06:09.428ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
121552025-12-16T03:06:09.428ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
121562025-12-16T03:06:09.428ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
121572025-12-16T03:06:09.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
121582025-12-16T03:06:09.431ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121592025-12-16T03:06:09.431ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121602025-12-16T03:06:09.432ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
121612025-12-16T03:06:09.432ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
121622025-12-16T03:06:09.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
121632025-12-16T03:06:09.433ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 162.675171ms iteration = 43 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
121642025-12-16T03:06:09.436ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
121652025-12-16T03:06:09.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
121662025-12-16T03:06:09.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
121672025-12-16T03:06:09.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
121682025-12-16T03:06:09.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
121692025-12-16T03:06:09.438ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
121702025-12-16T03:06:09.438ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
121712025-12-16T03:06:09.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
121722025-12-16T03:06:09.441ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
121732025-12-16T03:06:09.441ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
121742025-12-16T03:06:09.441ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-12-16 02:59:18.154440 UTC dns_group = external generation = 2
121752025-12-16T03:06:09.441ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 268.044556ms iteration = 8 status = {"generation":2}
121762025-12-16T03:06:09.441ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
121772025-12-16T03:06:09.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
121782025-12-16T03:06:09.463ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
121792025-12-16T03:06:09.463ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 192.500828ms iteration = 15 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
121802025-12-16T03:06:09.465ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
121812025-12-16T03:06:09.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
121822025-12-16T03:06:09.466ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
121832025-12-16T03:06:09.466ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
121842025-12-16T03:06:09.466ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
121852025-12-16T03:06:09.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
121862025-12-16T03:06:09.470ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
121872025-12-16T03:06:09.470ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
121882025-12-16T03:06:09.470ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 199.029291ms iteration = 15 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
121892025-12-16T03:06:09.505ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 233.868043ms iteration = 15 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
121902025-12-16T03:06:09.511ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
121912025-12-16T03:06:09.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
121922025-12-16T03:06:09.513ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121932025-12-16T03:06:09.514ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 340.99873ms iteration = 15 status = {}
121942025-12-16T03:06:09.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121952025-12-16T03:06:09.517ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
121962025-12-16T03:06:09.517ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
121972025-12-16T03:06:09.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
121982025-12-16T03:06:09.522ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
121992025-12-16T03:06:09.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122002025-12-16T03:06:09.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122012025-12-16T03:06:09.524ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
122022025-12-16T03:06:09.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
122032025-12-16T03:06:09.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122042025-12-16T03:06:09.530ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
122052025-12-16T03:06:09.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122062025-12-16T03:06:09.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122072025-12-16T03:06:09.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122082025-12-16T03:06:09.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122092025-12-16T03:06:09.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122102025-12-16T03:06:09.539ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
122112025-12-16T03:06:09.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
122122025-12-16T03:06:09.540ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
122132025-12-16T03:06:09.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122142025-12-16T03:06:09.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122152025-12-16T03:06:09.541ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122162025-12-16T03:06:09.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122172025-12-16T03:06:09.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122182025-12-16T03:06:09.543ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
122192025-12-16T03:06:09.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122202025-12-16T03:06:09.546ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 373.077247ms iteration = 15 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
122212025-12-16T03:06:09.546ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
122222025-12-16T03:06:09.546ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
122232025-12-16T03:06:09.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
122242025-12-16T03:06:09.549ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122252025-12-16T03:06:09.550ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
122262025-12-16T03:06:09.550ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122272025-12-16T03:06:09.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122282025-12-16T03:06:09.552ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122292025-12-16T03:06:09.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122302025-12-16T03:06:09.553ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
122312025-12-16T03:06:09.553ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
122322025-12-16T03:06:09.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
122332025-12-16T03:06:09.554ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122342025-12-16T03:06:09.560ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
122352025-12-16T03:06:09.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122362025-12-16T03:06:09.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122372025-12-16T03:06:09.574ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
122382025-12-16T03:06:09.576ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 305.313548ms iteration = 24 status = {"physical_disks_added":0}
122392025-12-16T03:06:09.580ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122402025-12-16T03:06:09.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122412025-12-16T03:06:09.583ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 01024e9a-d5fa-4122-8959-6a135f273dcd step component = DeployNexusRecords step id = Ensure
122422025-12-16T03:06:09.583ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
122432025-12-16T03:06:09.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
122442025-12-16T03:06:09.584ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122452025-12-16T03:06:09.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122462025-12-16T03:06:09.585ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122472025-12-16T03:06:09.586ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122482025-12-16T03:06:09.588ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
122492025-12-16T03:06:09.588ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
122502025-12-16T03:06:09.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
122512025-12-16T03:06:09.590ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
122522025-12-16T03:06:09.590ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
122532025-12-16T03:06:09.590ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122542025-12-16T03:06:09.590ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122552025-12-16T03:06:09.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
122562025-12-16T03:06:09.592ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122572025-12-16T03:06:09.592ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122582025-12-16T03:06:09.597ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
122592025-12-16T03:06:09.597ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122602025-12-16T03:06:09.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122612025-12-16T03:06:09.598ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
122622025-12-16T03:06:09.600ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122632025-12-16T03:06:09.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122642025-12-16T03:06:09.602ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122652025-12-16T03:06:09.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122662025-12-16T03:06:09.604ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122672025-12-16T03:06:09.608ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
122682025-12-16T03:06:09.608ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122692025-12-16T03:06:09.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122702025-12-16T03:06:09.610ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122712025-12-16T03:06:09.613ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
122722025-12-16T03:06:09.613ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122732025-12-16T03:06:09.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122742025-12-16T03:06:09.616ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122752025-12-16T03:06:09.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122762025-12-16T03:06:09.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122772025-12-16T03:06:09.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122782025-12-16T03:06:09.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122792025-12-16T03:06:09.625ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
122802025-12-16T03:06:09.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122812025-12-16T03:06:09.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122822025-12-16T03:06:09.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122832025-12-16T03:06:09.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122842025-12-16T03:06:09.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122852025-12-16T03:06:09.635ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
122862025-12-16T03:06:09.651ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
122872025-12-16T03:06:09.651ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 436.080665ms iteration = 8 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
122882025-12-16T03:06:09.660ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 01024e9a-d5fa-4122-8959-6a135f273dcd step component = SledAgent step id = Ensure
122892025-12-16T03:06:09.704ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122902025-12-16T03:06:09.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122912025-12-16T03:06:09.707ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122922025-12-16T03:06:09.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122932025-12-16T03:06:09.709ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122942025-12-16T03:06:09.712ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
122952025-12-16T03:06:09.713ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122962025-12-16T03:06:09.713ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122972025-12-16T03:06:09.714ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122982025-12-16T03:06:09.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122992025-12-16T03:06:09.716ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123002025-12-16T03:06:09.720ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
123012025-12-16T03:06:09.720ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123022025-12-16T03:06:09.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123032025-12-16T03:06:09.722ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123042025-12-16T03:06:09.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123052025-12-16T03:06:09.723ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123062025-12-16T03:06:09.724ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
123072025-12-16T03:06:09.725ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "49b8079f-b11d-42ac-889f-833242d962af", "date": "Tue, 16 Dec 2025 03:06:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
123082025-12-16T03:06:09.725ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
123092025-12-16T03:06:09.727ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
123102025-12-16T03:06:09.727ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123112025-12-16T03:06:09.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123122025-12-16T03:06:09.727ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "8e15bb35-a211-4747-8714-d03db28f0eb9", "date": "Tue, 16 Dec 2025 03:06:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
123132025-12-16T03:06:09.727ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
123142025-12-16T03:06:09.729ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123152025-12-16T03:06:09.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123162025-12-16T03:06:09.730ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
123172025-12-16T03:06:09.730ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 556.690353ms iteration = 10 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
123182025-12-16T03:06:09.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123192025-12-16T03:06:09.734ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
123202025-12-16T03:06:09.734ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123212025-12-16T03:06:09.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123222025-12-16T03:06:09.736ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
123232025-12-16T03:06:09.736ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
123242025-12-16T03:06:09.736ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3c3e5e4-69db-4e36-9605-e541b252d243", "content-length": "372", "date": "Tue, 16 Dec 2025 03:06:09 GMT"} })
123252025-12-16T03:06:09.736ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
123262025-12-16T03:06:09.736ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
123272025-12-16T03:06:09.736ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
123282025-12-16T03:06:09.736ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 425fc6c3-a3bb-4443-b264-5158cc9765a0 parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
123292025-12-16T03:06:09.736ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123302025-12-16T03:06:09.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
123312025-12-16T03:06:09.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123322025-12-16T03:06:09.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123332025-12-16T03:06:09.739ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
123342025-12-16T03:06:09.739ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
123352025-12-16T03:06:09.740ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "64dbc602-9bee-4020-af08-36abb1457292", "content-length": "26", "date": "Tue, 16 Dec 2025 03:06:09 GMT"} })
123362025-12-16T03:06:09.740ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
123372025-12-16T03:06:09.741ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
123382025-12-16T03:06:09.742ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
123392025-12-16T03:06:09.742ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123402025-12-16T03:06:09.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123412025-12-16T03:06:09.745ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
123422025-12-16T03:06:09.745ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
123432025-12-16T03:06:09.746ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 531.387602ms iteration = 19 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
123442025-12-16T03:06:09.751ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
123452025-12-16T03:06:09.751ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
123462025-12-16T03:06:09.752ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "d7ba511e-0714-4c0f-ab82-80afe4aa11ac", "date": "Tue, 16 Dec 2025 03:06:09 GMT"} })
123472025-12-16T03:06:09.752ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
123482025-12-16T03:06:09.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
123492025-12-16T03:06:09.754ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
123502025-12-16T03:06:09.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
123512025-12-16T03:06:09.758ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
123522025-12-16T03:06:09.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
123532025-12-16T03:06:09.761ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
123542025-12-16T03:06:09.761ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
123552025-12-16T03:06:09.764ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "821176ac-c735-4545-954c-e611f209dc04", "date": "Tue, 16 Dec 2025 03:06:09 GMT"} })
123562025-12-16T03:06:09.766ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 495.595899ms iteration = 22 status = {}
123572025-12-16T03:06:09.777ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
123582025-12-16T03:06:09.778ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "481837c9-72b6-4e3c-8e0d-34e781ddb105", "content-length": "691", "date": "Tue, 16 Dec 2025 03:06:09 GMT"} })
123592025-12-16T03:06:09.778ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
123602025-12-16T03:06:09.778ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
123612025-12-16T03:06:09.778ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
123622025-12-16T03:06:09.780ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 508.964559ms iteration = 15 status = {}
123632025-12-16T03:06:09.824ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
123642025-12-16T03:06:09.832ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "158f96f1-8868-4d02-899a-250b3a2a5bd7", "date": "Tue, 16 Dec 2025 03:06:09 GMT"} }) sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
123652025-12-16T03:06:09.834ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 01024e9a-d5fa-4122-8959-6a135f273dcd step component = FirewallRules step id = Ensure
123662025-12-16T03:06:09.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
123672025-12-16T03:06:09.834ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
123682025-12-16T03:06:09.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
123692025-12-16T03:06:09.834ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "9f4b2f3f-a60e-4790-b47e-74c4481fd42b", "date": "Tue, 16 Dec 2025 03:06:09 GMT"} })
123702025-12-16T03:06:09.834ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
123712025-12-16T03:06:09.836ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
123722025-12-16T03:06:09.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
123732025-12-16T03:06:09.837ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
123742025-12-16T03:06:09.837ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Dependency
123752025-12-16T03:06:09.837ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
123762025-12-16T03:06:09.837ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 605.596252ms iteration = 8 status = {"errors":[],"probes_by_sled":{"b1c76b61-acee-4be6-b582-973c938d6b6e":0}}
123772025-12-16T03:06:09.837ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
123782025-12-16T03:06:09.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
123792025-12-16T03:06:09.838ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
123802025-12-16T03:06:09.838ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
123812025-12-16T03:06:09.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
123822025-12-16T03:06:09.839ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "d8d6cdac-2f36-4d2a-b155-e1f17ec47496", "date": "Tue, 16 Dec 2025 03:06:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
123832025-12-16T03:06:09.839ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
123842025-12-16T03:06:09.839ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123852025-12-16T03:06:09.840ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
123862025-12-16T03:06:09.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
123872025-12-16T03:06:09.841ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "e0277d1c-1d59-4f9f-8577-4fc7a2ca19ac", "date": "Tue, 16 Dec 2025 03:06:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
123882025-12-16T03:06:09.841ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
123892025-12-16T03:06:09.841ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
123902025-12-16T03:06:09.841ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
123912025-12-16T03:06:09.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
123922025-12-16T03:06:09.842ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
123932025-12-16T03:06:09.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
123942025-12-16T03:06:09.843ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "548e552c-edcd-4f21-a444-9562d4876474", "date": "Tue, 16 Dec 2025 03:06:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
123952025-12-16T03:06:09.843ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
123962025-12-16T03:06:09.843ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123972025-12-16T03:06:09.844ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
123982025-12-16T03:06:09.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
123992025-12-16T03:06:09.845ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
124002025-12-16T03:06:09.845ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 671.507318ms iteration = 10 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
124012025-12-16T03:06:09.845ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124022025-12-16T03:06:09.846ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
124032025-12-16T03:06:09.846ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124042025-12-16T03:06:09.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124052025-12-16T03:06:09.846ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1377, tv_nsec: 458115374 } })
124062025-12-16T03:06:09.850ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
124072025-12-16T03:06:09.850ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124082025-12-16T03:06:09.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124092025-12-16T03:06:09.852ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124102025-12-16T03:06:09.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124112025-12-16T03:06:09.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124122025-12-16T03:06:09.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124132025-12-16T03:06:09.855ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124142025-12-16T03:06:09.859ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
124152025-12-16T03:06:09.859ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124162025-12-16T03:06:09.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124172025-12-16T03:06:09.863ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124182025-12-16T03:06:09.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124192025-12-16T03:06:09.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124202025-12-16T03:06:09.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124212025-12-16T03:06:09.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124222025-12-16T03:06:09.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124232025-12-16T03:06:09.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124242025-12-16T03:06:09.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124252025-12-16T03:06:09.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124262025-12-16T03:06:09.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124272025-12-16T03:06:09.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124282025-12-16T03:06:09.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124292025-12-16T03:06:09.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124302025-12-16T03:06:09.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
124312025-12-16T03:06:09.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124322025-12-16T03:06:09.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124332025-12-16T03:06:09.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124342025-12-16T03:06:09.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124352025-12-16T03:06:09.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124362025-12-16T03:06:09.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124372025-12-16T03:06:09.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124382025-12-16T03:06:09.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124392025-12-16T03:06:09.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124402025-12-16T03:06:09.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124412025-12-16T03:06:09.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124422025-12-16T03:06:09.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124432025-12-16T03:06:09.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124442025-12-16T03:06:09.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124452025-12-16T03:06:09.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124462025-12-16T03:06:09.897ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
124472025-12-16T03:06:09.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124482025-12-16T03:06:09.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124492025-12-16T03:06:09.899ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124502025-12-16T03:06:09.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124512025-12-16T03:06:09.901ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124522025-12-16T03:06:09.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124532025-12-16T03:06:09.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124542025-12-16T03:06:09.908ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
124552025-12-16T03:06:09.908ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124562025-12-16T03:06:09.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124572025-12-16T03:06:09.912ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124582025-12-16T03:06:09.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124592025-12-16T03:06:09.914ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124602025-12-16T03:06:09.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124612025-12-16T03:06:09.915ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
124622025-12-16T03:06:09.915ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
124632025-12-16T03:06:09.915ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1377, tv_nsec: 527575294 } })
124642025-12-16T03:06:09.916ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124652025-12-16T03:06:09.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124662025-12-16T03:06:09.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124672025-12-16T03:06:09.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124682025-12-16T03:06:09.920ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "0490a2ce-a05a-46b9-ad2c-bc8535268fdd", "content-length": "372", "date": "Tue, 16 Dec 2025 03:06:09 GMT"} })
124692025-12-16T03:06:09.920ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124702025-12-16T03:06:09.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124712025-12-16T03:06:09.922ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124722025-12-16T03:06:09.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124732025-12-16T03:06:09.924ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124742025-12-16T03:06:09.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124752025-12-16T03:06:09.926ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124762025-12-16T03:06:09.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124772025-12-16T03:06:09.928ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124782025-12-16T03:06:09.933ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
124792025-12-16T03:06:09.933ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124802025-12-16T03:06:09.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124812025-12-16T03:06:09.934ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
124822025-12-16T03:06:09.934ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
124832025-12-16T03:06:09.934ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124842025-12-16T03:06:09.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124852025-12-16T03:06:09.935ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "1cd8de6c-ea00-4a68-b622-f50d1b2e5ae6", "date": "Tue, 16 Dec 2025 03:06:09 GMT"} })
124862025-12-16T03:06:09.935ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124872025-12-16T03:06:09.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124882025-12-16T03:06:09.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124892025-12-16T03:06:09.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124902025-12-16T03:06:09.937ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124912025-12-16T03:06:09.938ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124922025-12-16T03:06:09.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124932025-12-16T03:06:09.942ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124942025-12-16T03:06:09.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124952025-12-16T03:06:09.943ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
124962025-12-16T03:06:09.943ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124972025-12-16T03:06:09.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124982025-12-16T03:06:09.945ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
124992025-12-16T03:06:09.945ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
125002025-12-16T03:06:09.946ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "692ab8c2-ea6f-46e7-b8cb-d079ab394868", "date": "Tue, 16 Dec 2025 03:06:09 GMT"} })
125012025-12-16T03:06:09.947ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
125022025-12-16T03:06:09.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
125032025-12-16T03:06:09.948ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 111.084098ms iteration = 23 status = {}
125042025-12-16T03:06:09.949ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
125052025-12-16T03:06:09.950ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
125062025-12-16T03:06:09.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
125072025-12-16T03:06:09.951ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }], Name("external-dns"): [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
125082025-12-16T03:06:09.951ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125092025-12-16T03:06:09.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125102025-12-16T03:06:09.953ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125112025-12-16T03:06:09.955ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
125122025-12-16T03:06:09.955ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125132025-12-16T03:06:09.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125142025-12-16T03:06:09.957ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
125152025-12-16T03:06:09.957ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
125162025-12-16T03:06:09.957ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
125172025-12-16T03:06:09.963ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
125182025-12-16T03:06:09.963ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
125192025-12-16T03:06:09.964ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
125202025-12-16T03:06:09.964ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
125212025-12-16T03:06:09.964ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
125222025-12-16T03:06:09.964ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
125232025-12-16T03:06:09.964ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
125242025-12-16T03:06:09.964ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
125252025-12-16T03:06:09.964ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
125262025-12-16T03:06:09.964ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
125272025-12-16T03:06:09.965ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
125282025-12-16T03:06:09.965ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
125292025-12-16T03:06:09.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
125302025-12-16T03:06:09.968ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
125312025-12-16T03:06:09.968ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
125322025-12-16T03:06:10.030ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), time_created: 2025-12-16T02:59:15.195809Z, time_modified: 2025-12-16T02:59:15.195809Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
125332025-12-16T03:06:10.030ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
125342025-12-16T03:06:10.062ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125352025-12-16T03:06:10.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125362025-12-16T03:06:10.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125372025-12-16T03:06:10.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125382025-12-16T03:06:10.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125392025-12-16T03:06:10.070ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
125402025-12-16T03:06:10.070ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
125412025-12-16T03:06:10.075ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "0f8fbafe-1558-4ef6-85cf-e966eb5ef6a8", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
125422025-12-16T03:06:10.076ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
125432025-12-16T03:06:10.076ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 01024e9a-d5fa-4122-8959-6a135f273dcd step component = Dns step id = Ensure
125442025-12-16T03:06:10.076ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
125452025-12-16T03:06:10.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125462025-12-16T03:06:10.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125472025-12-16T03:06:10.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125482025-12-16T03:06:10.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125492025-12-16T03:06:10.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125502025-12-16T03:06:10.082ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
125512025-12-16T03:06:10.083ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
125522025-12-16T03:06:10.083ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
125532025-12-16T03:06:10.083ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
125542025-12-16T03:06:10.083ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125552025-12-16T03:06:10.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125562025-12-16T03:06:10.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125572025-12-16T03:06:10.089ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
125582025-12-16T03:06:10.089ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125592025-12-16T03:06:10.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125602025-12-16T03:06:10.098ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "2aebb5df-92c9-48b7-9116-12140383502f", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
125612025-12-16T03:06:10.098ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
125622025-12-16T03:06:10.098ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
125632025-12-16T03:06:10.098ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca9661b1-ffec-4b8b-b8bc-db83a4c13204", "content-length": "682", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
125642025-12-16T03:06:10.099ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
125652025-12-16T03:06:10.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
125662025-12-16T03:06:10.102ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
125672025-12-16T03:06:10.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
125682025-12-16T03:06:10.105ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
125692025-12-16T03:06:10.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
125702025-12-16T03:06:10.108ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
125712025-12-16T03:06:10.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
125722025-12-16T03:06:10.117ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 846.247111ms iteration = 15 status = {}
125732025-12-16T03:06:10.129ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
125742025-12-16T03:06:10.129ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
125752025-12-16T03:06:10.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125762025-12-16T03:06:10.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125772025-12-16T03:06:10.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125782025-12-16T03:06:10.135ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
125792025-12-16T03:06:10.135ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125802025-12-16T03:06:10.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125812025-12-16T03:06:10.137ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
125822025-12-16T03:06:10.137ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
125832025-12-16T03:06:10.137ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
125842025-12-16T03:06:10.137ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125852025-12-16T03:06:10.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125862025-12-16T03:06:10.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125872025-12-16T03:06:10.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125882025-12-16T03:06:10.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125892025-12-16T03:06:10.143ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
125902025-12-16T03:06:10.144ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
125912025-12-16T03:06:10.144ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
125922025-12-16T03:06:10.144ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
125932025-12-16T03:06:10.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125942025-12-16T03:06:10.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125952025-12-16T03:06:10.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125962025-12-16T03:06:10.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
125972025-12-16T03:06:10.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125982025-12-16T03:06:10.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125992025-12-16T03:06:10.154ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
126002025-12-16T03:06:10.154ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
126012025-12-16T03:06:10.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126022025-12-16T03:06:10.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126032025-12-16T03:06:10.155ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126042025-12-16T03:06:10.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
126052025-12-16T03:06:10.159ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126062025-12-16T03:06:10.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126072025-12-16T03:06:10.160ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
126082025-12-16T03:06:10.160ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
126092025-12-16T03:06:10.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126102025-12-16T03:06:10.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126112025-12-16T03:06:10.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126122025-12-16T03:06:10.164ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
126132025-12-16T03:06:10.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126142025-12-16T03:06:10.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126152025-12-16T03:06:10.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126162025-12-16T03:06:10.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126172025-12-16T03:06:10.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126182025-12-16T03:06:10.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126192025-12-16T03:06:10.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126202025-12-16T03:06:10.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
126212025-12-16T03:06:10.173ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
126222025-12-16T03:06:10.173ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
126232025-12-16T03:06:10.173ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 01024e9a-d5fa-4122-8959-6a135f273dcd step component = OmicronZones step id = Cleanup
126242025-12-16T03:06:10.173ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 01024e9a-d5fa-4122-8959-6a135f273dcd step component = OmicronZones step id = Cleanup
126252025-12-16T03:06:10.173ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 01024e9a-d5fa-4122-8959-6a135f273dcd step component = PhysicalDisks step id = Cleanup
126262025-12-16T03:06:10.173ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 01024e9a-d5fa-4122-8959-6a135f273dcd step component = Clickhouse step id = Ensure
126272025-12-16T03:06:10.173ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 01024e9a-d5fa-4122-8959-6a135f273dcd step component = Clickhouse step id = Ensure
126282025-12-16T03:06:10.204ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
126292025-12-16T03:06:10.208ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "ca3dae0b-2f02-4ef3-8bf8-34411458b7b6", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
126302025-12-16T03:06:10.209ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 01024e9a-d5fa-4122-8959-6a135f273dcd step component = SupportBundles step id = Cleanup
126312025-12-16T03:06:10.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126322025-12-16T03:06:10.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126332025-12-16T03:06:10.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126342025-12-16T03:06:10.213ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
126352025-12-16T03:06:10.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126362025-12-16T03:06:10.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126372025-12-16T03:06:10.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126382025-12-16T03:06:10.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126392025-12-16T03:06:10.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126402025-12-16T03:06:10.219ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
126412025-12-16T03:06:10.260ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 01024e9a-d5fa-4122-8959-6a135f273dcd step component = OmicronZones step id = Cleanup
126422025-12-16T03:06:10.260ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
126432025-12-16T03:06:10.260ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
126442025-12-16T03:06:10.260ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126452025-12-16T03:06:10.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126462025-12-16T03:06:10.262ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126472025-12-16T03:06:10.264ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
126482025-12-16T03:06:10.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126492025-12-16T03:06:10.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126502025-12-16T03:06:10.302ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
126512025-12-16T03:06:10.302ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(a91734db-9ac7-4793-9c32-ddbb21f8d986 (blueprint), false)
126522025-12-16T03:06:10.302ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 01024e9a-d5fa-4122-8959-6a135f273dcd step component = Cockroach step id = Ensure
126532025-12-16T03:06:10.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126542025-12-16T03:06:10.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126552025-12-16T03:06:10.322ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
126562025-12-16T03:06:10.322ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 01024e9a-d5fa-4122-8959-6a135f273dcd step component = MgsUpdates step id = Ensure
126572025-12-16T03:06:10.322ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 9 reason = Dependency
126582025-12-16T03:06:10.322ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 23 reason = Dependency
126592025-12-16T03:06:10.322ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
126602025-12-16T03:06:10.322ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
126612025-12-16T03:06:10.322ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.148619693s iteration = 9 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"eaa0d58f-23ca-45e7-8dfc-a9b1da1ae8b4","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"eaa0d58f-23ca-45e7-8dfc-a9b1da1ae8b4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3970,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":67613267,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":67613227,"secs":0}},"event_index":1,"execution_id":"eaa0d58f-23ca-45e7-8dfc-a9b1da1ae8b4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":67631507,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":342142404,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":342142354,"secs":0}},"event_index":2,"execution_id":"eaa0d58f-23ca-45e7-8dfc-a9b1da1ae8b4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":409789611,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":76705092,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":76705052,"secs":0}},"event_index":3,"execution_id":"eaa0d58f-23ca-45e7-8dfc-a9b1da1ae8b4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":486517033,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":173934664,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":173934624,"secs":0}},"event_index":4,"execution_id":"eaa0d58f-23ca-45e7-8dfc-a9b1da1ae8b4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":660465617,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":242105598,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":242105558,"secs":0}},"event_index":5,"execution_id":"eaa0d58f-23ca-45e7-8dfc-a9b1da1ae8b4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":902582805,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":97642818,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":97642778,"secs":0}},"event_index":6,"execution_id":"eaa0d58f-23ca-45e7-8dfc-a9b1da1ae8b4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":234533,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4490,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4450,"secs":0}},"event_index":7,"execution_id":"eaa0d58f-23ca-45e7-8dfc-a9b1da1ae8b4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":251413,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2380,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2340,"secs":0}},"event_index":8,"execution_id":"eaa0d58f-23ca-45e7-8dfc-a9b1da1ae8b4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":256303,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2990,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":2960,"secs":0}},"event_index":9,"execution_id":"eaa0d58f-23ca-45e7-8dfc-a9b1da1ae8b4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":261203,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2100,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":2060,"secs":0}},"event_index":10,"execution_id":"eaa0d58f-23ca-45e7-8dfc-a9b1da1ae8b4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":264903,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":36012796,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":36012766,"secs":0}},"event_index":11,"execution_id":"eaa0d58f-23ca-45e7-8dfc-a9b1da1ae8b4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":36279809,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":50823014,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":50822974,"secs":0}},"event_index":12,"execution_id":"eaa0d58f-23ca-45e7-8dfc-a9b1da1ae8b4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":87114313,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":41732748,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":41732698,"secs":0}},"event_index":13,"execution_id":"eaa0d58f-23ca-45e7-8dfc-a9b1da1ae8b4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":128858472,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":19448917,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":19448887,"secs":0}},"event_index":14,"execution_id":"eaa0d58f-23ca-45e7-8dfc-a9b1da1ae8b4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":148315629,"secs":1}},{"data":{"attempt_elapsed":{"nanos":4600,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4570,"secs":0}},"event_index":15,"execution_id":"eaa0d58f-23ca-45e7-8dfc-a9b1da1ae8b4","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":148327239,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986"}
126622025-12-16T03:06:10.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
126632025-12-16T03:06:10.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
126642025-12-16T03:06:10.324ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126652025-12-16T03:06:10.324ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126662025-12-16T03:06:10.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
126672025-12-16T03:06:10.327ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
126682025-12-16T03:06:10.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
126692025-12-16T03:06:10.327ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
126702025-12-16T03:06:10.327ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
126712025-12-16T03:06:10.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
126722025-12-16T03:06:10.369ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
126732025-12-16T03:06:10.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
126742025-12-16T03:06:10.371ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126752025-12-16T03:06:10.373ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
126762025-12-16T03:06:10.373ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
126772025-12-16T03:06:10.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
126782025-12-16T03:06:10.375ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
126792025-12-16T03:06:10.375ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 53.374846ms iteration = 23 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
126802025-12-16T03:06:10.407ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(3c35439c-8f26-4169-b227-d3e30abe5b3b, 0), (200d7524-61d6-4d1f-9300-8156bef5d11c, 0), (b6c51ddf-9f27-432f-9d24-9fbc42795d0c, 0)]
126812025-12-16T03:06:10.408ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1378, tv_nsec: 20147575 } })
126822025-12-16T03:06:10.439ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("20488683-5778-4090-b36e-e6f0d80a682b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("35dbaad1-8536-44a0-8a0a-35d7c39dcfbb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b57fa226-2428-4a54-b85d-60409e28b484.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("d812a3e2-48ae-4812-a1cc-7eb7623e4e00.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::5)) }], valid_until: Instant { tv_sec: 1378, tv_nsec: 51627763 } })
126832025-12-16T03:06:10.440ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
126842025-12-16T03:06:10.440ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
126852025-12-16T03:06:10.440ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
126862025-12-16T03:06:10.440ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "6059e6ea-115f-4459-8c5a-c5254f4a70c4", "content-length": "476", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
126872025-12-16T03:06:10.440ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
126882025-12-16T03:06:10.441ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "09c005c0-a735-46a5-bb71-dc6d9007a8e0", "content-length": "734", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
126892025-12-16T03:06:10.441ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
126902025-12-16T03:06:10.441ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1555248a-8152-4923-9638-dee1c01dd031", "content-length": "111", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
126912025-12-16T03:06:10.441ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
126922025-12-16T03:06:10.442ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "435846af-3949-44e3-b021-fac056354528", "content-length": "111", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
126932025-12-16T03:06:10.442ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
126942025-12-16T03:06:10.442ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a051dc3-8b4c-4f33-b1c7-a88f3c3f8b4d", "content-length": "163", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
126952025-12-16T03:06:10.442ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
126962025-12-16T03:06:10.443ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7dfe429b-b6bc-41ac-86a6-34730fe9c0dc", "content-length": "163", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
126972025-12-16T03:06:10.443ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
126982025-12-16T03:06:10.443ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a93124e4-9e91-41c8-b267-d838bb9ca51e", "content-length": "166", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
126992025-12-16T03:06:10.443ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
127002025-12-16T03:06:10.443ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8db6af50-0fe2-4ac7-86b5-1cca81fa0ad2", "content-length": "166", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127012025-12-16T03:06:10.443ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
127022025-12-16T03:06:10.444ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4c8bbdb-56ce-4b62-8393-5ae860570321", "content-length": "702", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127032025-12-16T03:06:10.444ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
127042025-12-16T03:06:10.444ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4cfbd195-6903-45c4-b517-731866e70b7a", "content-length": "727", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127052025-12-16T03:06:10.444ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
127062025-12-16T03:06:10.444ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "62ffcef8-07d1-491f-8108-36eabf156a5c", "content-length": "729", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127072025-12-16T03:06:10.444ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
127082025-12-16T03:06:10.444ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "02ef0f07-5510-4852-94d5-dbaa44f35d38", "content-length": "728", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127092025-12-16T03:06:10.444ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
127102025-12-16T03:06:10.445ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e81730a-0af2-4735-a391-67cd8ca3899a", "content-length": "722", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127112025-12-16T03:06:10.445ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
127122025-12-16T03:06:10.445ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d4c1e7b-16c7-4c85-85f5-f314d639e482", "content-length": "10", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127132025-12-16T03:06:10.445ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
127142025-12-16T03:06:10.445ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9645c16a-d4c7-464b-bdc8-93cc5c2b9674", "content-length": "144", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127152025-12-16T03:06:10.445ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
127162025-12-16T03:06:10.445ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "466a8c2b-3339-41de-bb5e-3bdab3f5377b", "content-length": "147", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127172025-12-16T03:06:10.445ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
127182025-12-16T03:06:10.446ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3194842-12a0-4550-8ee6-2ac29f66116b", "content-length": "109", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127192025-12-16T03:06:10.446ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
127202025-12-16T03:06:10.446ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f71d8554-e8bd-4389-8b78-267a2f0dd0d8", "content-length": "109", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127212025-12-16T03:06:10.446ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
127222025-12-16T03:06:10.447ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "62c6092f-990a-498e-a0a6-0889088cc57f", "content-length": "163", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127232025-12-16T03:06:10.447ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
127242025-12-16T03:06:10.447ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a59d7f7-5a94-4267-94ac-679dff080977", "content-length": "163", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127252025-12-16T03:06:10.447ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
127262025-12-16T03:06:10.447ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "81d549c3-efe3-46c1-be6e-77f52f4cdb9d", "content-length": "166", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127272025-12-16T03:06:10.447ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
127282025-12-16T03:06:10.448ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7ac0a75-d909-4489-9222-f7cfe9a2820c", "content-length": "166", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127292025-12-16T03:06:10.448ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
127302025-12-16T03:06:10.448ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec1ceb61-f4ef-4bfb-a735-b48f1815ab84", "content-length": "702", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127312025-12-16T03:06:10.448ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
127322025-12-16T03:06:10.448ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ae0c768-35ad-4f7d-9ca6-fd3117b47339", "content-length": "727", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127332025-12-16T03:06:10.448ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
127342025-12-16T03:06:10.448ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbb2beba-68c2-4ff4-8fab-d22845c37bd5", "content-length": "729", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127352025-12-16T03:06:10.448ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
127362025-12-16T03:06:10.448ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5591e8c8-1b3b-4a78-89bc-6155ed28171b", "content-length": "728", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127372025-12-16T03:06:10.449ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
127382025-12-16T03:06:10.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
127392025-12-16T03:06:10.450ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127402025-12-16T03:06:10.453ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
127412025-12-16T03:06:10.453ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
127422025-12-16T03:06:10.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
127432025-12-16T03:06:10.454ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127442025-12-16T03:06:10.456ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
127452025-12-16T03:06:10.456ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
127462025-12-16T03:06:10.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
127472025-12-16T03:06:10.488ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
127482025-12-16T03:06:10.488ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
127492025-12-16T03:06:10.542ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "b21f9de0-7014-4c33-853d-91871ff98a55", "content-length": "95612", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127502025-12-16T03:06:10.544ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
127512025-12-16T03:06:10.544ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
127522025-12-16T03:06:10.544ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
127532025-12-16T03:06:10.544ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
127542025-12-16T03:06:10.544ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
127552025-12-16T03:06:10.544ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
127562025-12-16T03:06:10.544ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
127572025-12-16T03:06:10.544ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
127582025-12-16T03:06:10.544ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b2c0dc4-46da-4b3e-8eb7-f2f3fe400d5d", "content-length": "64", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127592025-12-16T03:06:10.544ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
127602025-12-16T03:06:10.544ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c8cb3d7-960f-4da4-a369-12d950a304f1", "content-length": "64", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127612025-12-16T03:06:10.544ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c12fc8bd-2367-4828-b201-59f2109c23e1", "content-length": "64", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127622025-12-16T03:06:10.544ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8093a45-fe3b-4cb1-9695-0b0ce876c481", "content-length": "64", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127632025-12-16T03:06:10.544ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
127642025-12-16T03:06:10.544ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "86074efc-e32f-4748-afe0-53a176e92c7f", "content-length": "64", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127652025-12-16T03:06:10.544ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
127662025-12-16T03:06:10.544ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
127672025-12-16T03:06:10.544ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
127682025-12-16T03:06:10.586ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "9dae26dd-80df-45c7-b54f-bceb46e5cee2", "content-length": "402499", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127692025-12-16T03:06:10.587ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "57394bb9-ae2b-4d32-8426-72edf7cdb37b", "content-length": "402225", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127702025-12-16T03:06:10.587ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "d110cc7a-275b-428b-9289-abcd9fd6986f", "content-length": "430847", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127712025-12-16T03:06:10.590ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "befd06f5-8247-4893-b678-1e4626c6c856", "content-length": "400245", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127722025-12-16T03:06:10.592ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "75c7a715-d0ee-45f5-93f0-df0edff1dbf0", "content-length": "401055", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127732025-12-16T03:06:10.595ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
127742025-12-16T03:06:10.595ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
127752025-12-16T03:06:10.595ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
127762025-12-16T03:06:10.595ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
127772025-12-16T03:06:10.595ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
127782025-12-16T03:06:10.628ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c (omicron_zone)
127792025-12-16T03:06:10.628ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
127802025-12-16T03:06:10.631ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "5798ff32-32c5-4fd3-a8b4-4f53d456f7ed", "content-length": "124", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127812025-12-16T03:06:10.633ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
127822025-12-16T03:06:10.726ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
127832025-12-16T03:06:10.726ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
127842025-12-16T03:06:10.727ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "787de2d7-1608-47f5-92c7-3d18d7768d9c", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127852025-12-16T03:06:10.727ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
127862025-12-16T03:06:10.728ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
127872025-12-16T03:06:10.728ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
127882025-12-16T03:06:10.728ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e6248ca-3e1f-45fe-8b10-8c442aecc341", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127892025-12-16T03:06:10.729ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
127902025-12-16T03:06:10.729ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
127912025-12-16T03:06:10.729ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
127922025-12-16T03:06:10.730ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c135b8c1-790c-432d-a898-0a378399bb44", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:06:10 GMT"} })
127932025-12-16T03:06:10.730ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
127942025-12-16T03:06:10.732ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
127952025-12-16T03:06:10.732ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
127962025-12-16T03:06:10.733ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
127972025-12-16T03:06:10.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
127982025-12-16T03:06:10.735ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127992025-12-16T03:06:10.738ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
128002025-12-16T03:06:10.738ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
128012025-12-16T03:06:10.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
128022025-12-16T03:06:10.949ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = d36b7584-5102-4a4f-888b-fa1c6fea6614 file = nexus/db-queries/src/db/datastore/inventory.rs:1613
128032025-12-16T03:06:10.949ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = d36b7584-5102-4a4f-888b-fa1c6fea6614 time_started = 2025-12-16 03:06:10.440485 UTC
128042025-12-16T03:06:10.949ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 38 reason = Dependency
128052025-12-16T03:06:10.949ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
128062025-12-16T03:06:10.949ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 26 reason = Dependency
128072025-12-16T03:06:10.949ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
128082025-12-16T03:06:10.949ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 19.041\u{b5}s iteration = 26 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
128092025-12-16T03:06:10.949ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 627.811824ms iteration = 9 status = {"collection_id":"d36b7584-5102-4a4f-888b-fa1c6fea6614","time_done":"2025-12-16 03:06:10.732076 UTC","time_started":"2025-12-16 03:06:10.440485 UTC"}
128102025-12-16T03:06:10.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
128112025-12-16T03:06:10.951ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128122025-12-16T03:06:10.955ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
128132025-12-16T03:06:10.955ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
128142025-12-16T03:06:10.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
128152025-12-16T03:06:11.065ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 317 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 55539f44-a089-472f-b4f2-2e47a41b552b response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
128162025-12-16T03:06:11.105ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
128172025-12-16T03:06:11.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
128182025-12-16T03:06:11.154ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 12 reason = Dependency
128192025-12-16T03:06:11.154ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 204.43028ms iteration = 38 status = {"Loaded":{"collection_id":"b97ed3f8-6719-4b04-8f47-be78e07de798","time_loaded":"2025-12-16T03:06:10.949860893Z","time_started":"2025-12-16T03:06:10.485695Z"}}
128202025-12-16T03:06:11.154ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 27 reason = Dependency
128212025-12-16T03:06:11.154ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
128222025-12-16T03:06:11.154ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Dependency
128232025-12-16T03:06:11.154ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 7.78\u{b5}s iteration = 27 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
128242025-12-16T03:06:11.154ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 20 reason = Dependency
128252025-12-16T03:06:11.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
128262025-12-16T03:06:11.154ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
128272025-12-16T03:06:11.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
128282025-12-16T03:06:11.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
128292025-12-16T03:06:11.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
128302025-12-16T03:06:11.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
128312025-12-16T03:06:11.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128322025-12-16T03:06:11.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128332025-12-16T03:06:11.157ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128342025-12-16T03:06:11.159ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
128352025-12-16T03:06:11.159ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
128362025-12-16T03:06:11.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
128372025-12-16T03:06:11.160ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
128382025-12-16T03:06:11.160ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
128392025-12-16T03:06:11.160ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
128402025-12-16T03:06:11.160ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
128412025-12-16T03:06:11.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
128422025-12-16T03:06:11.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
128432025-12-16T03:06:11.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128442025-12-16T03:06:11.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128452025-12-16T03:06:11.162ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
128462025-12-16T03:06:11.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
128472025-12-16T03:06:11.164ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128482025-12-16T03:06:11.165ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
128492025-12-16T03:06:11.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
128502025-12-16T03:06:11.165ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
128512025-12-16T03:06:11.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
128522025-12-16T03:06:11.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
128532025-12-16T03:06:11.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
128542025-12-16T03:06:11.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
128552025-12-16T03:06:11.167ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
128562025-12-16T03:06:11.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
128572025-12-16T03:06:11.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
128582025-12-16T03:06:11.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
128592025-12-16T03:06:11.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
128602025-12-16T03:06:11.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
128612025-12-16T03:06:11.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128622025-12-16T03:06:11.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128632025-12-16T03:06:11.172ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
128642025-12-16T03:06:11.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
128652025-12-16T03:06:11.172ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
128662025-12-16T03:06:11.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
128672025-12-16T03:06:11.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
128682025-12-16T03:06:11.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
128692025-12-16T03:06:11.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128702025-12-16T03:06:11.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128712025-12-16T03:06:11.176ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
128722025-12-16T03:06:11.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
128732025-12-16T03:06:11.176ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
128742025-12-16T03:06:11.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
128752025-12-16T03:06:11.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
128762025-12-16T03:06:11.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
128772025-12-16T03:06:11.178ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
128782025-12-16T03:06:11.178ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
128792025-12-16T03:06:11.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
128802025-12-16T03:06:11.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
128812025-12-16T03:06:11.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
128822025-12-16T03:06:11.180ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128832025-12-16T03:06:11.181ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
128842025-12-16T03:06:11.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
128852025-12-16T03:06:11.182ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
128862025-12-16T03:06:11.182ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
128872025-12-16T03:06:11.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
128882025-12-16T03:06:11.183ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128892025-12-16T03:06:11.183ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128902025-12-16T03:06:11.186ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
128912025-12-16T03:06:11.186ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
128922025-12-16T03:06:11.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
128932025-12-16T03:06:11.187ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
128942025-12-16T03:06:11.187ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
128952025-12-16T03:06:11.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
128962025-12-16T03:06:11.189ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
128972025-12-16T03:06:11.189ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 34.812581ms iteration = 12 status = {"blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","inventory_collection_id":"b97ed3f8-6719-4b04-8f47-be78e07de798","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
128982025-12-16T03:06:11.189ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
128992025-12-16T03:06:11.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129002025-12-16T03:06:11.190ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129012025-12-16T03:06:11.193ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
129022025-12-16T03:06:11.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129032025-12-16T03:06:11.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129042025-12-16T03:06:11.195ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129052025-12-16T03:06:11.198ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
129062025-12-16T03:06:11.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129072025-12-16T03:06:11.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129082025-12-16T03:06:11.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129092025-12-16T03:06:11.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129102025-12-16T03:06:11.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129112025-12-16T03:06:11.207ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
129122025-12-16T03:06:11.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129132025-12-16T03:06:11.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129142025-12-16T03:06:11.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129152025-12-16T03:06:11.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129162025-12-16T03:06:11.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129172025-12-16T03:06:11.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
129182025-12-16T03:06:11.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129192025-12-16T03:06:11.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129202025-12-16T03:06:11.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129212025-12-16T03:06:11.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129222025-12-16T03:06:11.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129232025-12-16T03:06:11.220ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 66.036117ms iteration = 25 status = {"physical_disks_added":0}
129242025-12-16T03:06:11.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
129252025-12-16T03:06:11.221ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129262025-12-16T03:06:11.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129272025-12-16T03:06:11.234ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129282025-12-16T03:06:11.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129292025-12-16T03:06:11.236ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129302025-12-16T03:06:11.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129312025-12-16T03:06:11.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129322025-12-16T03:06:11.241ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
129332025-12-16T03:06:11.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129342025-12-16T03:06:11.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129352025-12-16T03:06:11.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129362025-12-16T03:06:11.245ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
129372025-12-16T03:06:11.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129382025-12-16T03:06:11.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129392025-12-16T03:06:11.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129402025-12-16T03:06:11.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129412025-12-16T03:06:11.249ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129422025-12-16T03:06:11.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129432025-12-16T03:06:11.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129442025-12-16T03:06:11.254ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
129452025-12-16T03:06:11.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129462025-12-16T03:06:11.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129472025-12-16T03:06:11.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129482025-12-16T03:06:11.258ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
129492025-12-16T03:06:11.258ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129502025-12-16T03:06:11.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129512025-12-16T03:06:11.261ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129522025-12-16T03:06:11.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129532025-12-16T03:06:11.261ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129542025-12-16T03:06:11.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129552025-12-16T03:06:11.263ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129562025-12-16T03:06:11.266ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
129572025-12-16T03:06:11.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129582025-12-16T03:06:11.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129592025-12-16T03:06:11.268ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129602025-12-16T03:06:11.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129612025-12-16T03:06:11.269ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129622025-12-16T03:06:11.273ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
129632025-12-16T03:06:11.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129642025-12-16T03:06:11.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129652025-12-16T03:06:11.311ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129662025-12-16T03:06:11.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129672025-12-16T03:06:11.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129682025-12-16T03:06:11.317ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
129692025-12-16T03:06:11.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129702025-12-16T03:06:11.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129712025-12-16T03:06:11.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129722025-12-16T03:06:11.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129732025-12-16T03:06:11.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129742025-12-16T03:06:11.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
129752025-12-16T03:06:11.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129762025-12-16T03:06:11.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129772025-12-16T03:06:11.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129782025-12-16T03:06:11.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129792025-12-16T03:06:11.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129802025-12-16T03:06:11.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
129812025-12-16T03:06:11.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129822025-12-16T03:06:11.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129832025-12-16T03:06:11.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129842025-12-16T03:06:11.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129852025-12-16T03:06:11.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129862025-12-16T03:06:11.336ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
129872025-12-16T03:06:11.336ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129882025-12-16T03:06:11.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129892025-12-16T03:06:11.373ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
129902025-12-16T03:06:11.373ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
129912025-12-16T03:06:11.373ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 01cf2e33-278c-4881-ad58-a926d3029c0b parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
129922025-12-16T03:06:11.373ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129932025-12-16T03:06:11.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129942025-12-16T03:06:11.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129952025-12-16T03:06:11.378ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
129962025-12-16T03:06:11.378ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129972025-12-16T03:06:11.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129982025-12-16T03:06:11.380ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
129992025-12-16T03:06:11.380ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
130002025-12-16T03:06:11.380ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 226.104655ms iteration = 20 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
130012025-12-16T03:06:14.271ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 86 reason = Timeout
130022025-12-16T03:06:14.271ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
130032025-12-16T03:06:14.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
130042025-12-16T03:06:14.273ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130052025-12-16T03:06:14.276ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
130062025-12-16T03:06:14.276ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
130072025-12-16T03:06:14.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
130082025-12-16T03:06:14.279ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
130092025-12-16T03:06:14.279ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.820849ms iteration = 86 status = {"config_updated":false}
130102025-12-16T03:06:19.271ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
130112025-12-16T03:06:19.271ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 44 reason = Timeout
130122025-12-16T03:06:19.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
130132025-12-16T03:06:19.271ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
130142025-12-16T03:06:19.271ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 87 reason = Timeout
130152025-12-16T03:06:19.271ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
130162025-12-16T03:06:19.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
130172025-12-16T03:06:19.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
130182025-12-16T03:06:19.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
130192025-12-16T03:06:19.273ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130202025-12-16T03:06:19.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130212025-12-16T03:06:19.274ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
130222025-12-16T03:06:19.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
130232025-12-16T03:06:19.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
130242025-12-16T03:06:19.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
130252025-12-16T03:06:19.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
130262025-12-16T03:06:19.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
130272025-12-16T03:06:19.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
130282025-12-16T03:06:19.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
130292025-12-16T03:06:19.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
130302025-12-16T03:06:19.279ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
130312025-12-16T03:06:19.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
130322025-12-16T03:06:19.279ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.316566ms iteration = 87 status = {"config_updated":false}
130332025-12-16T03:06:19.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130342025-12-16T03:06:19.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
130352025-12-16T03:06:19.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
130362025-12-16T03:06:19.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
130372025-12-16T03:06:19.297ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
130382025-12-16T03:06:19.297ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.666995ms iteration = 44 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
130392025-12-16T03:06:19.312ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 41.03425ms iteration = 44 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
130402025-12-16T03:06:21.058ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 24c7947b-1ddf-4833-a18a-243ac7ae463c uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending version = 1.0.0
130412025-12-16T03:06:21.058ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 24c7947b-1ddf-4833-a18a-243ac7ae463c roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
130422025-12-16T03:06:21.059ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 24c7947b-1ddf-4833-a18a-243ac7ae463c resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
130432025-12-16T03:06:21.062ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 3870 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 24c7947b-1ddf-4833-a18a-243ac7ae463c response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending
130442025-12-16T03:06:21.062ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 89035e9b-32f0-4926-bce7-c6cf3e5c1d4c uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D version = 1.0.0
130452025-12-16T03:06:21.063ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 89035e9b-32f0-4926-bce7-c6cf3e5c1d4c roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
130462025-12-16T03:06:21.063ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 89035e9b-32f0-4926-bce7-c6cf3e5c1d4c resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
130472025-12-16T03:06:21.065ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 2146 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 89035e9b-32f0-4926-bce7-c6cf3e5c1d4c response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D
130482025-12-16T03:06:21.065ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 272 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 96f56036-56bc-4fa0-8eb2-6f886db17fad response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
130492025-12-16T03:06:24.271ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 39 reason = Timeout
130502025-12-16T03:06:24.271ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
130512025-12-16T03:06:24.271ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 30 reason = Timeout
130522025-12-16T03:06:24.271ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 88 reason = Timeout
130532025-12-16T03:06:24.271ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
130542025-12-16T03:06:24.272ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
130552025-12-16T03:06:24.272ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 30 reason = Timeout
130562025-12-16T03:06:24.272ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
130572025-12-16T03:06:24.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
130582025-12-16T03:06:24.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
130592025-12-16T03:06:24.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
130602025-12-16T03:06:24.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
130612025-12-16T03:06:24.273ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
130622025-12-16T03:06:24.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
130632025-12-16T03:06:24.273ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130642025-12-16T03:06:24.274ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130652025-12-16T03:06:24.274ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130662025-12-16T03:06:24.274ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130672025-12-16T03:06:24.277ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
130682025-12-16T03:06:24.278ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
130692025-12-16T03:06:24.278ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
130702025-12-16T03:06:24.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
130712025-12-16T03:06:24.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
130722025-12-16T03:06:24.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
130732025-12-16T03:06:24.278ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
130742025-12-16T03:06:24.279ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
130752025-12-16T03:06:24.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
130762025-12-16T03:06:24.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
130772025-12-16T03:06:24.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
130782025-12-16T03:06:24.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
130792025-12-16T03:06:24.279ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
130802025-12-16T03:06:24.279ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 7.95192ms iteration = 30 status = "NoSitrep"
130812025-12-16T03:06:24.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
130822025-12-16T03:06:24.280ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = b97ed3f8-6719-4b04-8f47-be78e07de798 original_time_started = 2025-12-16 03:06:10.485695 UTC
130832025-12-16T03:06:24.280ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.696901ms iteration = 39 status = {"Loaded":{"collection_id":"b97ed3f8-6719-4b04-8f47-be78e07de798","time_loaded":"2025-12-16T03:06:24.271859591Z","time_started":"2025-12-16T03:06:10.485695Z"}}
130842025-12-16T03:06:24.280ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
130852025-12-16T03:06:24.280ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.756201ms iteration = 88 status = {"config_updated":false}
130862025-12-16T03:06:24.280ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130872025-12-16T03:06:24.282ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
130882025-12-16T03:06:24.282ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
130892025-12-16T03:06:24.282ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 11.05728ms iteration = 30 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
130902025-12-16T03:06:29.271ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
130912025-12-16T03:06:29.271ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 45 reason = Timeout
130922025-12-16T03:06:29.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
130932025-12-16T03:06:29.271ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
130942025-12-16T03:06:29.271ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 89 reason = Timeout
130952025-12-16T03:06:29.271ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
130962025-12-16T03:06:29.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
130972025-12-16T03:06:29.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
130982025-12-16T03:06:29.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
130992025-12-16T03:06:29.273ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131002025-12-16T03:06:29.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131012025-12-16T03:06:29.274ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
131022025-12-16T03:06:29.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
131032025-12-16T03:06:29.275ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.499937ms iteration = 45 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
131042025-12-16T03:06:29.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
131052025-12-16T03:06:29.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
131062025-12-16T03:06:29.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
131072025-12-16T03:06:29.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
131082025-12-16T03:06:29.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
131092025-12-16T03:06:29.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
131102025-12-16T03:06:29.280ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
131112025-12-16T03:06:29.280ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 9.103625ms iteration = 89 status = {"config_updated":false}
131122025-12-16T03:06:29.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
131132025-12-16T03:06:29.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
131142025-12-16T03:06:29.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131152025-12-16T03:06:29.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
131162025-12-16T03:06:29.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
131172025-12-16T03:06:29.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
131182025-12-16T03:06:29.297ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
131192025-12-16T03:06:29.297ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.361464ms iteration = 45 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
131202025-12-16T03:06:31.066ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 436 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 0040fb13-81a6-4498-808a-17b516e8e4bd response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
131212025-12-16T03:06:34.272ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 90 reason = Timeout
131222025-12-16T03:06:34.272ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
131232025-12-16T03:06:34.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
131242025-12-16T03:06:34.273ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131252025-12-16T03:06:34.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
131262025-12-16T03:06:34.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
131272025-12-16T03:06:34.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
131282025-12-16T03:06:34.279ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
131292025-12-16T03:06:34.279ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.234202ms iteration = 90 status = {"config_updated":false}
131302025-12-16T03:06:39.173ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
131312025-12-16T03:06:39.173ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
131322025-12-16T03:06:39.173ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
131332025-12-16T03:06:39.173ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 16 reason = Timeout
131342025-12-16T03:06:39.173ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
131352025-12-16T03:06:39.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
131362025-12-16T03:06:39.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
131372025-12-16T03:06:39.176ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1406, tv_nsec: 787645759 } })
131382025-12-16T03:06:39.176ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1406, tv_nsec: 787995414 } })
131392025-12-16T03:06:39.177ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
131402025-12-16T03:06:39.177ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1406, tv_nsec: 788637062 } })
131412025-12-16T03:06:39.177ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.960271ms iteration = 16 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
131422025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 16 reason = Timeout
131432025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 21.381\u{b5}s iteration = 16 status = {"disabled":true,"errors":[],"sps":[]}
131442025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 16 reason = Timeout
131452025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 40 reason = Timeout
131462025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 31 reason = Timeout
131472025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
131482025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 16 reason = Timeout
131492025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
131502025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
131512025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
131522025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
131532025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
131542025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 24 reason = Timeout
131552025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 16 reason = Timeout
131562025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
131572025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
131582025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 16 reason = Timeout
131592025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
131602025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 16 reason = Timeout
131612025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
131622025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
131632025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 16 reason = Timeout
131642025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
131652025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 46 reason = Timeout
131662025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
131672025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 16 reason = Timeout
131682025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
131692025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
131702025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Timeout
131712025-12-16T03:06:39.272ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
131722025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
131732025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
131742025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 16 reason = Timeout
131752025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
131762025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
131772025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
131782025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
131792025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
131802025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 91 reason = Timeout
131812025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
131822025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
131832025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
131842025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 31 reason = Timeout
131852025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
131862025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
131872025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
131882025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
131892025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
131902025-12-16T03:06:39.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
131912025-12-16T03:06:39.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
131922025-12-16T03:06:39.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
131932025-12-16T03:06:39.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
131942025-12-16T03:06:39.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
131952025-12-16T03:06:39.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
131962025-12-16T03:06:39.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
131972025-12-16T03:06:39.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
131982025-12-16T03:06:39.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
131992025-12-16T03:06:39.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
132002025-12-16T03:06:39.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
132012025-12-16T03:06:39.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
132022025-12-16T03:06:39.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
132032025-12-16T03:06:39.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
132042025-12-16T03:06:39.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
132052025-12-16T03:06:39.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
132062025-12-16T03:06:39.274ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
132072025-12-16T03:06:39.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
132082025-12-16T03:06:39.274ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132092025-12-16T03:06:39.275ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132102025-12-16T03:06:39.275ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132112025-12-16T03:06:39.275ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132122025-12-16T03:06:39.275ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132132025-12-16T03:06:39.275ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
132142025-12-16T03:06:39.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132152025-12-16T03:06:39.276ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132162025-12-16T03:06:39.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
132172025-12-16T03:06:39.277ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132182025-12-16T03:06:39.277ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132192025-12-16T03:06:39.279ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 7.320673ms iteration = 16 status = {"errors":[],"requests_created_ok":[]}
132202025-12-16T03:06:39.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
132212025-12-16T03:06:39.281ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
132222025-12-16T03:06:39.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
132232025-12-16T03:06:39.281ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
132242025-12-16T03:06:39.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
132252025-12-16T03:06:39.281ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
132262025-12-16T03:06:39.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
132272025-12-16T03:06:39.281ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
132282025-12-16T03:06:39.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
132292025-12-16T03:06:39.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
132302025-12-16T03:06:39.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
132312025-12-16T03:06:39.282ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
132322025-12-16T03:06:39.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
132332025-12-16T03:06:39.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
132342025-12-16T03:06:39.283ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
132352025-12-16T03:06:39.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
132362025-12-16T03:06:39.283ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
132372025-12-16T03:06:39.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
132382025-12-16T03:06:39.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
132392025-12-16T03:06:39.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132402025-12-16T03:06:39.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
132412025-12-16T03:06:39.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
132422025-12-16T03:06:39.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
132432025-12-16T03:06:39.283ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
132442025-12-16T03:06:39.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
132452025-12-16T03:06:39.284ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
132462025-12-16T03:06:39.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
132472025-12-16T03:06:39.285ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
132482025-12-16T03:06:39.285ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
132492025-12-16T03:06:39.285ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 12.750502ms iteration = 91 status = {"config_updated":false}
132502025-12-16T03:06:39.285ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1406, tv_nsec: 896645701 } })
132512025-12-16T03:06:39.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
132522025-12-16T03:06:39.285ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = b97ed3f8-6719-4b04-8f47-be78e07de798 original_time_started = 2025-12-16 03:06:10.485695 UTC
132532025-12-16T03:06:39.285ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 13.598092ms iteration = 40 status = {"Loaded":{"collection_id":"b97ed3f8-6719-4b04-8f47-be78e07de798","time_loaded":"2025-12-16T03:06:39.272255683Z","time_started":"2025-12-16T03:06:10.485695Z"}}
132542025-12-16T03:06:39.285ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
132552025-12-16T03:06:39.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132562025-12-16T03:06:39.286ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
132572025-12-16T03:06:39.286ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132582025-12-16T03:06:39.286ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1406, tv_nsec: 897529642 } })
132592025-12-16T03:06:39.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
132602025-12-16T03:06:39.286ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
132612025-12-16T03:06:39.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
132622025-12-16T03:06:39.286ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 13.897756ms iteration = 16 status = 0
132632025-12-16T03:06:39.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
132642025-12-16T03:06:39.287ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132652025-12-16T03:06:39.288ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132662025-12-16T03:06:39.289ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
132672025-12-16T03:06:39.289ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 17.022376ms iteration = 16 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
132682025-12-16T03:06:39.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
132692025-12-16T03:06:39.290ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
132702025-12-16T03:06:39.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
132712025-12-16T03:06:39.290ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
132722025-12-16T03:06:39.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
132732025-12-16T03:06:39.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
132742025-12-16T03:06:39.291ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
132752025-12-16T03:06:39.291ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
132762025-12-16T03:06:39.291ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 19.099132ms iteration = 31 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
132772025-12-16T03:06:39.292ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
132782025-12-16T03:06:39.292ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 20.54442ms iteration = 24 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
132792025-12-16T03:06:39.302ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 30.244683ms iteration = 26 status = {"physical_disks_added":0}
132802025-12-16T03:06:39.313ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
132812025-12-16T03:06:39.313ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
132822025-12-16T03:06:39.313ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "11ef18e3-eed7-4a79-95cb-10a188a19247", "content-length": "26", "date": "Tue, 16 Dec 2025 03:06:39 GMT"} })
132832025-12-16T03:06:39.313ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
132842025-12-16T03:06:39.315ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
132852025-12-16T03:06:39.316ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1406, tv_nsec: 927564852 } })
132862025-12-16T03:06:39.324ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
132872025-12-16T03:06:39.326ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 152.822966ms iteration = 16 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
132882025-12-16T03:06:39.327ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132892025-12-16T03:06:39.331ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
132902025-12-16T03:06:39.331ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
132912025-12-16T03:06:39.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
132922025-12-16T03:06:39.332ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
132932025-12-16T03:06:39.332ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
132942025-12-16T03:06:39.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
132952025-12-16T03:06:39.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
132962025-12-16T03:06:39.334ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132972025-12-16T03:06:39.334ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 61.966475ms iteration = 31 status = "NoSitrep"
132982025-12-16T03:06:39.334ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
132992025-12-16T03:06:39.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
133002025-12-16T03:06:39.337ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 65.169826ms iteration = 16 status = {"errors":[],"garbage_collect_requested":[]}
133012025-12-16T03:06:39.337ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
133022025-12-16T03:06:39.337ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
133032025-12-16T03:06:39.337ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 65.347398ms iteration = 16 status = {"errors":[],"finish_invoked_ok":[]}
133042025-12-16T03:06:39.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
133052025-12-16T03:06:39.338ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
133062025-12-16T03:06:39.338ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
133072025-12-16T03:06:39.338ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
133082025-12-16T03:06:39.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
133092025-12-16T03:06:39.339ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133102025-12-16T03:06:39.339ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
133112025-12-16T03:06:39.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
133122025-12-16T03:06:39.339ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
133132025-12-16T03:06:39.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
133142025-12-16T03:06:39.340ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 68.440557ms iteration = 46 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
133152025-12-16T03:06:39.341ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
133162025-12-16T03:06:39.341ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
133172025-12-16T03:06:39.341ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
133182025-12-16T03:06:39.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
133192025-12-16T03:06:39.341ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
133202025-12-16T03:06:39.341ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
133212025-12-16T03:06:39.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
133222025-12-16T03:06:39.343ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
133232025-12-16T03:06:39.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
133242025-12-16T03:06:39.353ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
133252025-12-16T03:06:39.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
133262025-12-16T03:06:39.355ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133272025-12-16T03:06:39.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133282025-12-16T03:06:39.357ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
133292025-12-16T03:06:39.357ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
133302025-12-16T03:06:39.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
133312025-12-16T03:06:39.359ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
133322025-12-16T03:06:39.359ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
133332025-12-16T03:06:39.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
133342025-12-16T03:06:39.359ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133352025-12-16T03:06:39.361ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
133362025-12-16T03:06:39.361ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
133372025-12-16T03:06:39.361ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "96aadaa8-9e81-4026-bb20-1a140041cd59", "content-length": "26", "date": "Tue, 16 Dec 2025 03:06:39 GMT"} })
133382025-12-16T03:06:39.361ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
133392025-12-16T03:06:39.362ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
133402025-12-16T03:06:39.362ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
133412025-12-16T03:06:39.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
133422025-12-16T03:06:39.363ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
133432025-12-16T03:06:39.372ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
133442025-12-16T03:06:39.372ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 99.828325ms iteration = 46 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
133452025-12-16T03:06:39.383ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 111.541783ms iteration = 16 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
133462025-12-16T03:06:39.393ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 120.968943ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
133472025-12-16T03:06:39.393ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 220.353102ms iteration = 16 status = {}
133482025-12-16T03:06:39.396ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 124.189684ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
133492025-12-16T03:06:39.399ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
133502025-12-16T03:06:39.399ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
133512025-12-16T03:06:39.399ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 127.476105ms iteration = 16 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
133522025-12-16T03:06:39.413ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1407, tv_nsec: 24776204 } })
133532025-12-16T03:06:39.453ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1407, tv_nsec: 64490576 } })
133542025-12-16T03:06:39.525ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
133552025-12-16T03:06:39.525ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
133562025-12-16T03:06:39.526ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a63998f-8f30-4e83-974d-cda3fb1a397b", "content-length": "372", "date": "Tue, 16 Dec 2025 03:06:39 GMT"} })
133572025-12-16T03:06:39.526ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
133582025-12-16T03:06:39.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
133592025-12-16T03:06:39.536ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
133602025-12-16T03:06:39.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
133612025-12-16T03:06:39.539ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
133622025-12-16T03:06:39.539ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
133632025-12-16T03:06:39.539ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
133642025-12-16T03:06:39.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
133652025-12-16T03:06:39.540ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
133662025-12-16T03:06:39.540ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
133672025-12-16T03:06:39.540ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "19acb019-7992-496c-b038-718d7f1db61f", "content-length": "691", "date": "Tue, 16 Dec 2025 03:06:39 GMT"} })
133682025-12-16T03:06:39.540ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
133692025-12-16T03:06:39.540ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
133702025-12-16T03:06:39.540ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
133712025-12-16T03:06:39.540ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "315b530f-f4c3-4b2a-a9c3-9d568fb6af7c", "date": "Tue, 16 Dec 2025 03:06:39 GMT"} })
133722025-12-16T03:06:39.540ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
133732025-12-16T03:06:39.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
133742025-12-16T03:06:39.541ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 269.431003ms iteration = 16 status = {}
133752025-12-16T03:06:39.543ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
133762025-12-16T03:06:39.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
133772025-12-16T03:06:39.546ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
133782025-12-16T03:06:39.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
133792025-12-16T03:06:39.549ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
133802025-12-16T03:06:39.549ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
133812025-12-16T03:06:39.551ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
133822025-12-16T03:06:39.551ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
133832025-12-16T03:06:39.552ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
133842025-12-16T03:06:39.552ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
133852025-12-16T03:06:39.552ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
133862025-12-16T03:06:39.552ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
133872025-12-16T03:06:39.552ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
133882025-12-16T03:06:39.552ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
133892025-12-16T03:06:39.552ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
133902025-12-16T03:06:39.552ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
133912025-12-16T03:06:39.552ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
133922025-12-16T03:06:39.552ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
133932025-12-16T03:06:39.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
133942025-12-16T03:06:39.553ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "5d4885d1-9700-4c7f-b2c8-784d2683133e", "date": "Tue, 16 Dec 2025 03:06:39 GMT"} })
133952025-12-16T03:06:39.554ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
133962025-12-16T03:06:39.555ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
133972025-12-16T03:06:39.555ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 282.684582ms iteration = 24 status = {}
133982025-12-16T03:06:39.709ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "1c53dc3e-d64f-4a74-ae53-6df432e4a9f8", "date": "Tue, 16 Dec 2025 03:06:39 GMT"} })
133992025-12-16T03:06:39.709ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
134002025-12-16T03:06:39.709ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
134012025-12-16T03:06:39.710ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "dba5142e-1f72-4656-9b74-a67d90b4c2ea", "content-length": "682", "date": "Tue, 16 Dec 2025 03:06:39 GMT"} })
134022025-12-16T03:06:39.710ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
134032025-12-16T03:06:39.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
134042025-12-16T03:06:39.714ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
134052025-12-16T03:06:39.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
134062025-12-16T03:06:39.717ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
134072025-12-16T03:06:39.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
134082025-12-16T03:06:39.719ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
134092025-12-16T03:06:39.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
134102025-12-16T03:06:39.726ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 453.815429ms iteration = 16 status = {}
134112025-12-16T03:06:41.066ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 311 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = b5150ea8-9d6f-4f44-9c86-887b93e5ce51 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
134122025-12-16T03:06:44.271ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 92 reason = Timeout
134132025-12-16T03:06:44.271ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
134142025-12-16T03:06:44.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
134152025-12-16T03:06:44.274ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134162025-12-16T03:06:44.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
134172025-12-16T03:06:44.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
134182025-12-16T03:06:44.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
134192025-12-16T03:06:44.279ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
134202025-12-16T03:06:44.279ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.91644ms iteration = 92 status = {"config_updated":false}
134212025-12-16T03:06:49.271ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 47 reason = Timeout
134222025-12-16T03:06:49.271ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
134232025-12-16T03:06:49.271ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
134242025-12-16T03:06:49.271ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 93 reason = Timeout
134252025-12-16T03:06:49.272ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
134262025-12-16T03:06:49.272ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
134272025-12-16T03:06:49.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
134282025-12-16T03:06:49.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
134292025-12-16T03:06:49.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
134302025-12-16T03:06:49.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134312025-12-16T03:06:49.274ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134322025-12-16T03:06:49.274ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
134332025-12-16T03:06:49.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
134342025-12-16T03:06:49.276ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.611258ms iteration = 47 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
134352025-12-16T03:06:49.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
134362025-12-16T03:06:49.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
134372025-12-16T03:06:49.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
134382025-12-16T03:06:49.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
134392025-12-16T03:06:49.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
134402025-12-16T03:06:49.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
134412025-12-16T03:06:49.280ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
134422025-12-16T03:06:49.280ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.468878ms iteration = 93 status = {"config_updated":false}
134432025-12-16T03:06:49.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
134442025-12-16T03:06:49.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
134452025-12-16T03:06:49.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134462025-12-16T03:06:49.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
134472025-12-16T03:06:49.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
134482025-12-16T03:06:49.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
134492025-12-16T03:06:49.296ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
134502025-12-16T03:06:49.296ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.639083ms iteration = 47 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
134512025-12-16T03:06:51.059ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 58516342-2749-43d4-acf4-5a1fb5f33b30 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending version = 1.0.0
134522025-12-16T03:06:51.059ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 58516342-2749-43d4-acf4-5a1fb5f33b30 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
134532025-12-16T03:06:51.059ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 58516342-2749-43d4-acf4-5a1fb5f33b30 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
134542025-12-16T03:06:51.061ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 2480 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 58516342-2749-43d4-acf4-5a1fb5f33b30 response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending
134552025-12-16T03:06:51.062ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = bb9591fe-8ca4-400d-a5b2-8141c0ec7727 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D version = 1.0.0
134562025-12-16T03:06:51.062ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = bb9591fe-8ca4-400d-a5b2-8141c0ec7727 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
134572025-12-16T03:06:51.062ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = bb9591fe-8ca4-400d-a5b2-8141c0ec7727 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
134582025-12-16T03:06:51.064ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 2511 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = bb9591fe-8ca4-400d-a5b2-8141c0ec7727 response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D
134592025-12-16T03:06:51.066ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 266 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 6e2ae0fb-951f-453a-8870-400f9e37a92a response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
134602025-12-16T03:06:54.272ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 41 reason = Timeout
134612025-12-16T03:06:54.272ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 32 reason = Timeout
134622025-12-16T03:06:54.272ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
134632025-12-16T03:06:54.272ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
134642025-12-16T03:06:54.272ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 94 reason = Timeout
134652025-12-16T03:06:54.272ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 32 reason = Timeout
134662025-12-16T03:06:54.272ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
134672025-12-16T03:06:54.272ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
134682025-12-16T03:06:54.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
134692025-12-16T03:06:54.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
134702025-12-16T03:06:54.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
134712025-12-16T03:06:54.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
134722025-12-16T03:06:54.273ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
134732025-12-16T03:06:54.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
134742025-12-16T03:06:54.274ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134752025-12-16T03:06:54.274ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134762025-12-16T03:06:54.274ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134772025-12-16T03:06:54.275ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134782025-12-16T03:06:54.278ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
134792025-12-16T03:06:54.278ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
134802025-12-16T03:06:54.278ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
134812025-12-16T03:06:54.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
134822025-12-16T03:06:54.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
134832025-12-16T03:06:54.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
134842025-12-16T03:06:54.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
134852025-12-16T03:06:54.279ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
134862025-12-16T03:06:54.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
134872025-12-16T03:06:54.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
134882025-12-16T03:06:54.280ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 8.014202ms iteration = 32 status = "NoSitrep"
134892025-12-16T03:06:54.280ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = b97ed3f8-6719-4b04-8f47-be78e07de798 original_time_started = 2025-12-16 03:06:10.485695 UTC
134902025-12-16T03:06:54.280ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.364006ms iteration = 41 status = {"Loaded":{"collection_id":"b97ed3f8-6719-4b04-8f47-be78e07de798","time_loaded":"2025-12-16T03:06:54.272288950Z","time_started":"2025-12-16T03:06:10.485695Z"}}
134912025-12-16T03:06:54.281ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
134922025-12-16T03:06:54.281ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
134932025-12-16T03:06:54.281ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 9.034855ms iteration = 94 status = {"config_updated":false}
134942025-12-16T03:06:54.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
134952025-12-16T03:06:54.282ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
134962025-12-16T03:06:54.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
134972025-12-16T03:06:54.283ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134982025-12-16T03:06:54.285ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
134992025-12-16T03:06:54.285ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
135002025-12-16T03:06:54.285ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 13.546502ms iteration = 32 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
135012025-12-16T03:06:59.272ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 48 reason = Timeout
135022025-12-16T03:06:59.272ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
135032025-12-16T03:06:59.272ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
135042025-12-16T03:06:59.272ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
135052025-12-16T03:06:59.272ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 95 reason = Timeout
135062025-12-16T03:06:59.272ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
135072025-12-16T03:06:59.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
135082025-12-16T03:06:59.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
135092025-12-16T03:06:59.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
135102025-12-16T03:06:59.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135112025-12-16T03:06:59.274ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135122025-12-16T03:06:59.274ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
135132025-12-16T03:06:59.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
135142025-12-16T03:06:59.276ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.298365ms iteration = 48 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
135152025-12-16T03:06:59.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
135162025-12-16T03:06:59.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
135172025-12-16T03:06:59.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
135182025-12-16T03:06:59.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
135192025-12-16T03:06:59.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
135202025-12-16T03:06:59.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
135212025-12-16T03:06:59.279ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
135222025-12-16T03:06:59.279ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.346193ms iteration = 95 status = {"config_updated":false}
135232025-12-16T03:06:59.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
135242025-12-16T03:06:59.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
135252025-12-16T03:06:59.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135262025-12-16T03:06:59.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
135272025-12-16T03:06:59.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
135282025-12-16T03:06:59.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
135292025-12-16T03:06:59.297ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
135302025-12-16T03:06:59.297ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.187869ms iteration = 48 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
135312025-12-16T03:07:01.067ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 369 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 712bdd51-d26c-4622-8978-00ea3f20c3a5 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
135322025-12-16T03:07:04.272ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 96 reason = Timeout
135332025-12-16T03:07:04.272ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
135342025-12-16T03:07:04.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
135352025-12-16T03:07:04.274ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135362025-12-16T03:07:04.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
135372025-12-16T03:07:04.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
135382025-12-16T03:07:04.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
135392025-12-16T03:07:04.279ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
135402025-12-16T03:07:04.279ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.194571ms iteration = 96 status = {"config_updated":false}
135412025-12-16T03:07:06.059ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 8ca8d244-a7a7-4a6b-83b4-6fb9487a9114 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending version = 1.0.0
135422025-12-16T03:07:06.059ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 8ca8d244-a7a7-4a6b-83b4-6fb9487a9114 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
135432025-12-16T03:07:06.059ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 8ca8d244-a7a7-4a6b-83b4-6fb9487a9114 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
135442025-12-16T03:07:06.062ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 3352 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 8ca8d244-a7a7-4a6b-83b4-6fb9487a9114 response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending
135452025-12-16T03:07:06.062ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 707388ea-7e3b-4669-8654-1324b0122df5 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D version = 1.0.0
135462025-12-16T03:07:06.062ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 707388ea-7e3b-4669-8654-1324b0122df5 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
135472025-12-16T03:07:06.063ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 707388ea-7e3b-4669-8654-1324b0122df5 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
135482025-12-16T03:07:06.065ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 2403 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 707388ea-7e3b-4669-8654-1324b0122df5 response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D
135492025-12-16T03:07:09.174ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
135502025-12-16T03:07:09.174ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
135512025-12-16T03:07:09.174ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
135522025-12-16T03:07:09.174ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
135532025-12-16T03:07:09.174ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
135542025-12-16T03:07:09.174ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
135552025-12-16T03:07:09.174ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
135562025-12-16T03:07:09.174ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
135572025-12-16T03:07:09.174ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
135582025-12-16T03:07:09.174ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
135592025-12-16T03:07:09.174ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
135602025-12-16T03:07:09.174ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 17 reason = Timeout
135612025-12-16T03:07:09.174ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
135622025-12-16T03:07:09.174ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
135632025-12-16T03:07:09.174ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
135642025-12-16T03:07:09.174ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
135652025-12-16T03:07:09.174ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
135662025-12-16T03:07:09.174ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
135672025-12-16T03:07:09.174ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ec7f925f-4c14-415c-9476-8dd4dc57451d step component = ExternalNetworking step id = Ensure
135682025-12-16T03:07:09.174ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
135692025-12-16T03:07:09.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
135702025-12-16T03:07:09.174ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
135712025-12-16T03:07:09.174ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 9 reason = Timeout
135722025-12-16T03:07:09.174ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-12-16 02:57:09.174722111 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
135732025-12-16T03:07:09.174ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
135742025-12-16T03:07:09.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
135752025-12-16T03:07:09.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
135762025-12-16T03:07:09.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
135772025-12-16T03:07:09.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
135782025-12-16T03:07:09.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
135792025-12-16T03:07:09.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
135802025-12-16T03:07:09.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
135812025-12-16T03:07:09.175ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1a45cd23-3d0a-43eb-adce-62c7b712baf1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1436, tv_nsec: 786100450 } })
135822025-12-16T03:07:09.175ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("49a51a59-9919-48ae-8182-4070a8430160.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6af60386-8771-4401-9d4d-e93774e519f0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 1436, tv_nsec: 786205121 } })
135832025-12-16T03:07:09.175ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
135842025-12-16T03:07:09.175ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.239746ms iteration = 9 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
135852025-12-16T03:07:09.176ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
135862025-12-16T03:07:09.176ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.378347ms iteration = 9 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
135872025-12-16T03:07:09.176ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
135882025-12-16T03:07:09.176ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
135892025-12-16T03:07:09.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
135902025-12-16T03:07:09.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
135912025-12-16T03:07:09.177ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1436, tv_nsec: 788287208 } })
135922025-12-16T03:07:09.177ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135932025-12-16T03:07:09.177ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
135942025-12-16T03:07:09.178ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-12-16 02:57:09.174722111 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
135952025-12-16T03:07:09.178ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.338312ms iteration = 9 status = {"expiration":"2025-12-16T02:57:09.174722111Z","pruned":[]}
135962025-12-16T03:07:09.178ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135972025-12-16T03:07:09.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
135982025-12-16T03:07:09.178ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135992025-12-16T03:07:09.178ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1436, tv_nsec: 789610244 } })
136002025-12-16T03:07:09.179ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
136012025-12-16T03:07:09.179ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.565808ms iteration = 17 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
136022025-12-16T03:07:09.179ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1436, tv_nsec: 790273703 } })
136032025-12-16T03:07:09.180ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
136042025-12-16T03:07:09.180ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
136052025-12-16T03:07:09.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
136062025-12-16T03:07:09.181ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136072025-12-16T03:07:09.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136082025-12-16T03:07:09.183ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
136092025-12-16T03:07:09.183ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
136102025-12-16T03:07:09.183ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136112025-12-16T03:07:09.184ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
136122025-12-16T03:07:09.184ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
136132025-12-16T03:07:09.184ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
136142025-12-16T03:07:09.184ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
136152025-12-16T03:07:09.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
136162025-12-16T03:07:09.184ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
136172025-12-16T03:07:09.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
136182025-12-16T03:07:09.185ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136192025-12-16T03:07:09.188ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
136202025-12-16T03:07:09.190ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136212025-12-16T03:07:09.191ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
136222025-12-16T03:07:09.193ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
136232025-12-16T03:07:09.193ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
136242025-12-16T03:07:09.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
136252025-12-16T03:07:09.195ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
136262025-12-16T03:07:09.195ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
136272025-12-16T03:07:09.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
136282025-12-16T03:07:09.196ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
136292025-12-16T03:07:09.196ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
136302025-12-16T03:07:09.196ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
136312025-12-16T03:07:09.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
136322025-12-16T03:07:09.200ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136332025-12-16T03:07:09.203ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
136342025-12-16T03:07:09.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
136352025-12-16T03:07:09.205ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
136362025-12-16T03:07:09.205ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
136372025-12-16T03:07:09.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
136382025-12-16T03:07:09.206ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136392025-12-16T03:07:09.208ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
136402025-12-16T03:07:09.208ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
136412025-12-16T03:07:09.208ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-12-16 02:59:18.154440 UTC dns_group = external generation = 2
136422025-12-16T03:07:09.208ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 34.208533ms iteration = 9 status = {"generation":2}
136432025-12-16T03:07:09.209ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
136442025-12-16T03:07:09.209ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
136452025-12-16T03:07:09.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
136462025-12-16T03:07:09.214ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
136472025-12-16T03:07:09.215ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 21 reason = Timeout
136482025-12-16T03:07:09.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
136492025-12-16T03:07:09.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
136502025-12-16T03:07:09.216ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 9 reason = Timeout
136512025-12-16T03:07:09.216ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 9 reason = Timeout
136522025-12-16T03:07:09.216ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
136532025-12-16T03:07:09.216ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
136542025-12-16T03:07:09.216ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
136552025-12-16T03:07:09.216ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 9 reason = Timeout
136562025-12-16T03:07:09.216ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
136572025-12-16T03:07:09.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
136582025-12-16T03:07:09.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
136592025-12-16T03:07:09.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
136602025-12-16T03:07:09.217ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
136612025-12-16T03:07:09.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136622025-12-16T03:07:09.221ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136632025-12-16T03:07:09.221ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
136642025-12-16T03:07:09.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
136652025-12-16T03:07:09.222ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
136662025-12-16T03:07:09.223ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
136672025-12-16T03:07:09.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
136682025-12-16T03:07:09.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
136692025-12-16T03:07:09.224ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
136702025-12-16T03:07:09.224ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
136712025-12-16T03:07:09.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
136722025-12-16T03:07:09.224ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
136732025-12-16T03:07:09.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136742025-12-16T03:07:09.226ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
136752025-12-16T03:07:09.226ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
136762025-12-16T03:07:09.226ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 9.771563ms iteration = 9 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
136772025-12-16T03:07:09.227ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
136782025-12-16T03:07:09.229ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
136792025-12-16T03:07:09.229ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
136802025-12-16T03:07:09.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
136812025-12-16T03:07:09.229ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
136822025-12-16T03:07:09.231ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
136832025-12-16T03:07:09.232ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
136842025-12-16T03:07:09.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
136852025-12-16T03:07:09.233ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
136862025-12-16T03:07:09.233ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 28 reason = Timeout
136872025-12-16T03:07:09.233ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 9 reason = Timeout
136882025-12-16T03:07:09.233ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
136892025-12-16T03:07:09.233ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
136902025-12-16T03:07:09.233ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 9 reason = Timeout
136912025-12-16T03:07:09.233ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
136922025-12-16T03:07:09.233ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 24.401\u{b5}s iteration = 28 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
136932025-12-16T03:07:09.233ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 9 reason = Timeout
136942025-12-16T03:07:09.233ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
136952025-12-16T03:07:09.233ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 9 reason = Timeout
136962025-12-16T03:07:09.233ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
136972025-12-16T03:07:09.233ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
136982025-12-16T03:07:09.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
136992025-12-16T03:07:09.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
137002025-12-16T03:07:09.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
137012025-12-16T03:07:09.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
137022025-12-16T03:07:09.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137032025-12-16T03:07:09.235ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
137042025-12-16T03:07:09.236ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137052025-12-16T03:07:09.237ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
137062025-12-16T03:07:09.237ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 3.478354ms iteration = 9 status = {"by_rx":{},"error":null}
137072025-12-16T03:07:09.237ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
137082025-12-16T03:07:09.238ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.602198ms iteration = 9 status = {"errors":[],"found_port_ok":[]}
137092025-12-16T03:07:09.239ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
137102025-12-16T03:07:09.239ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
137112025-12-16T03:07:09.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
137122025-12-16T03:07:09.240ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ec7f925f-4c14-415c-9476-8dd4dc57451d step component = SledList step id = Fetch
137132025-12-16T03:07:09.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
137142025-12-16T03:07:09.240ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
137152025-12-16T03:07:09.240ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
137162025-12-16T03:07:09.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
137172025-12-16T03:07:09.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
137182025-12-16T03:07:09.241ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
137192025-12-16T03:07:09.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
137202025-12-16T03:07:09.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
137212025-12-16T03:07:09.242ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 8.282885ms iteration = 9 status = {"dispatched":[],"errors":[],"glob_version":"212.0.0","globs_reprocessed":{},"no_receivers":[]}
137222025-12-16T03:07:09.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137232025-12-16T03:07:09.247ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
137242025-12-16T03:07:09.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
137252025-12-16T03:07:09.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
137262025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 17 reason = Timeout
137272025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 17 reason = Timeout
137282025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
137292025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 27.97\u{b5}s iteration = 17 status = {"disabled":true,"errors":[],"sps":[]}
137302025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 17 reason = Timeout
137312025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
137322025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
137332025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
137342025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
137352025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 17 reason = Timeout
137362025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 17 reason = Timeout
137372025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
137382025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
137392025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
137402025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
137412025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 17 reason = Timeout
137422025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
137432025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 42 reason = Timeout
137442025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 17 reason = Timeout
137452025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
137462025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
137472025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 49 reason = Timeout
137482025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
137492025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
137502025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
137512025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 17 reason = Timeout
137522025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
137532025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 25 reason = Timeout
137542025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
137552025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 33 reason = Timeout
137562025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
137572025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
137582025-12-16T03:07:09.272ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 97 reason = Timeout
137592025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
137602025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
137612025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Timeout
137622025-12-16T03:07:09.273ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
137632025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
137642025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
137652025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
137662025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 33 reason = Timeout
137672025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
137682025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 17 reason = Timeout
137692025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
137702025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
137712025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
137722025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
137732025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
137742025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
137752025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
137762025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
137772025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
137782025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
137792025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
137802025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
137812025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
137822025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
137832025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
137842025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
137852025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
137862025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
137872025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
137882025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
137892025-12-16T03:07:09.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
137902025-12-16T03:07:09.276ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
137912025-12-16T03:07:09.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
137922025-12-16T03:07:09.276ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137932025-12-16T03:07:09.277ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
137942025-12-16T03:07:09.277ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
137952025-12-16T03:07:09.277ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
137962025-12-16T03:07:09.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
137972025-12-16T03:07:09.279ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137982025-12-16T03:07:09.280ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
137992025-12-16T03:07:09.280ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
138002025-12-16T03:07:09.280ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
138012025-12-16T03:07:09.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
138022025-12-16T03:07:09.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
138032025-12-16T03:07:09.284ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
138042025-12-16T03:07:09.284ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
138052025-12-16T03:07:09.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
138062025-12-16T03:07:09.294ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
138072025-12-16T03:07:09.294ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
138082025-12-16T03:07:09.294ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
138092025-12-16T03:07:09.294ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
138102025-12-16T03:07:09.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
138112025-12-16T03:07:09.297ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138122025-12-16T03:07:09.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
138132025-12-16T03:07:09.301ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
138142025-12-16T03:07:09.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
138152025-12-16T03:07:09.302ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138162025-12-16T03:07:09.304ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-12-16T02:59:05.020237Z, time_modified: 2025-12-16T02:59:05.020237Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }) }
138172025-12-16T03:07:09.304ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 130.238751ms iteration = 9 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]}},"default_endpoint":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
138182025-12-16T03:07:09.307ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
138192025-12-16T03:07:09.308ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
138202025-12-16T03:07:09.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
138212025-12-16T03:07:09.308ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138222025-12-16T03:07:09.312ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
138232025-12-16T03:07:09.316ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
138242025-12-16T03:07:09.316ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
138252025-12-16T03:07:09.316ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
138262025-12-16T03:07:09.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
138272025-12-16T03:07:09.320ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
138282025-12-16T03:07:09.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
138292025-12-16T03:07:09.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
138302025-12-16T03:07:09.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
138312025-12-16T03:07:09.332ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138322025-12-16T03:07:09.332ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
138332025-12-16T03:07:09.332ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
138342025-12-16T03:07:09.332ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 116.214122ms iteration = 9 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
138352025-12-16T03:07:09.336ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
138362025-12-16T03:07:09.336ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
138372025-12-16T03:07:09.336ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
138382025-12-16T03:07:09.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
138392025-12-16T03:07:09.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
138402025-12-16T03:07:09.337ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138412025-12-16T03:07:09.339ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = b97ed3f8-6719-4b04-8f47-be78e07de798 original_time_started = 2025-12-16 03:06:10.485695 UTC
138422025-12-16T03:07:09.339ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 66.994749ms iteration = 42 status = {"Loaded":{"collection_id":"b97ed3f8-6719-4b04-8f47-be78e07de798","time_loaded":"2025-12-16T03:07:09.272731243Z","time_started":"2025-12-16T03:06:10.485695Z"}}
138432025-12-16T03:07:09.343ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
138442025-12-16T03:07:09.343ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
138452025-12-16T03:07:09.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
138462025-12-16T03:07:09.344ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138472025-12-16T03:07:09.346ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
138482025-12-16T03:07:09.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
138492025-12-16T03:07:09.348ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138502025-12-16T03:07:09.349ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
138512025-12-16T03:07:09.349ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
138522025-12-16T03:07:09.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
138532025-12-16T03:07:09.351ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138542025-12-16T03:07:09.353ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
138552025-12-16T03:07:09.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
138562025-12-16T03:07:09.353ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138572025-12-16T03:07:09.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
138582025-12-16T03:07:09.355ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
138592025-12-16T03:07:09.355ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
138602025-12-16T03:07:09.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
138612025-12-16T03:07:09.357ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
138622025-12-16T03:07:09.357ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
138632025-12-16T03:07:09.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
138642025-12-16T03:07:09.359ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138652025-12-16T03:07:09.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138662025-12-16T03:07:09.361ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138672025-12-16T03:07:09.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138682025-12-16T03:07:09.363ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
138692025-12-16T03:07:09.363ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
138702025-12-16T03:07:09.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
138712025-12-16T03:07:09.364ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 91.579441ms iteration = 17 status = {"errors":[],"requests_created_ok":[]}
138722025-12-16T03:07:09.364ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
138732025-12-16T03:07:09.365ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
138742025-12-16T03:07:09.365ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 92.637324ms iteration = 33 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
138752025-12-16T03:07:09.366ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138762025-12-16T03:07:09.369ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
138772025-12-16T03:07:09.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
138782025-12-16T03:07:09.370ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
138792025-12-16T03:07:09.370ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
138802025-12-16T03:07:09.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
138812025-12-16T03:07:09.372ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138822025-12-16T03:07:09.372ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
138832025-12-16T03:07:09.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
138842025-12-16T03:07:09.376ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138852025-12-16T03:07:09.379ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
138862025-12-16T03:07:09.379ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
138872025-12-16T03:07:09.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
138882025-12-16T03:07:09.379ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138892025-12-16T03:07:09.379ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
138902025-12-16T03:07:09.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
138912025-12-16T03:07:09.381ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1436, tv_nsec: 991893027 } })
138922025-12-16T03:07:09.381ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
138932025-12-16T03:07:09.381ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
138942025-12-16T03:07:09.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138952025-12-16T03:07:09.381ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 108.386984ms iteration = 33 status = "NoSitrep"
138962025-12-16T03:07:09.381ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
138972025-12-16T03:07:09.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
138982025-12-16T03:07:09.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
138992025-12-16T03:07:09.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139002025-12-16T03:07:09.381ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
139012025-12-16T03:07:09.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
139022025-12-16T03:07:09.382ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1436, tv_nsec: 992966921 } })
139032025-12-16T03:07:09.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
139042025-12-16T03:07:09.382ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
139052025-12-16T03:07:09.382ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 109.72109ms iteration = 97 status = {"config_updated":false}
139062025-12-16T03:07:09.383ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
139072025-12-16T03:07:09.383ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
139082025-12-16T03:07:09.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
139092025-12-16T03:07:09.385ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 112.657108ms iteration = 49 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
139102025-12-16T03:07:09.386ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 113.586319ms iteration = 17 status = {"errors":[],"finish_invoked_ok":[]}
139112025-12-16T03:07:09.387ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
139122025-12-16T03:07:09.387ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
139132025-12-16T03:07:09.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
139142025-12-16T03:07:09.388ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
139152025-12-16T03:07:09.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139162025-12-16T03:07:09.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139172025-12-16T03:07:09.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
139182025-12-16T03:07:09.391ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139192025-12-16T03:07:09.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
139202025-12-16T03:07:09.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139212025-12-16T03:07:09.393ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139222025-12-16T03:07:09.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139232025-12-16T03:07:09.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139242025-12-16T03:07:09.395ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
139252025-12-16T03:07:09.395ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
139262025-12-16T03:07:09.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
139272025-12-16T03:07:09.396ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 123.94484ms iteration = 17 status = {"errors":[],"garbage_collect_requested":[]}
139282025-12-16T03:07:09.399ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
139292025-12-16T03:07:09.399ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
139302025-12-16T03:07:09.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
139312025-12-16T03:07:09.401ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
139322025-12-16T03:07:09.401ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139332025-12-16T03:07:09.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139342025-12-16T03:07:09.403ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139352025-12-16T03:07:09.403ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
139362025-12-16T03:07:09.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
139372025-12-16T03:07:09.408ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
139382025-12-16T03:07:09.408ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139392025-12-16T03:07:09.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139402025-12-16T03:07:09.409ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
139412025-12-16T03:07:09.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139422025-12-16T03:07:09.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139432025-12-16T03:07:09.413ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139442025-12-16T03:07:09.414ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 239.425664ms iteration = 17 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
139452025-12-16T03:07:09.414ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139462025-12-16T03:07:09.416ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
139472025-12-16T03:07:09.418ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
139482025-12-16T03:07:09.418ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
139492025-12-16T03:07:09.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
139502025-12-16T03:07:09.419ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
139512025-12-16T03:07:09.419ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139522025-12-16T03:07:09.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139532025-12-16T03:07:09.424ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 152.007766ms iteration = 17 status = 0
139542025-12-16T03:07:09.425ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139552025-12-16T03:07:09.425ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
139562025-12-16T03:07:09.426ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
139572025-12-16T03:07:09.426ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 153.382774ms iteration = 17 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
139582025-12-16T03:07:09.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139592025-12-16T03:07:09.428ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
139602025-12-16T03:07:09.428ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 155.649933ms iteration = 49 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
139612025-12-16T03:07:09.428ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139622025-12-16T03:07:09.433ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
139632025-12-16T03:07:09.433ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139642025-12-16T03:07:09.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139652025-12-16T03:07:09.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139662025-12-16T03:07:09.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139672025-12-16T03:07:09.439ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139682025-12-16T03:07:09.441ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
139692025-12-16T03:07:09.441ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
139702025-12-16T03:07:09.441ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
139712025-12-16T03:07:09.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
139722025-12-16T03:07:09.444ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139732025-12-16T03:07:09.444ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
139742025-12-16T03:07:09.444ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139752025-12-16T03:07:09.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139762025-12-16T03:07:09.445ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
139772025-12-16T03:07:09.446ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 173.162343ms iteration = 25 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
139782025-12-16T03:07:09.449ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
139792025-12-16T03:07:09.449ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
139802025-12-16T03:07:09.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
139812025-12-16T03:07:09.450ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
139822025-12-16T03:07:09.450ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
139832025-12-16T03:07:09.451ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c908db41-0a0e-49dd-bb60-e2fd56cae1f5", "content-length": "26", "date": "Tue, 16 Dec 2025 03:07:09 GMT"} })
139842025-12-16T03:07:09.451ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
139852025-12-16T03:07:09.453ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
139862025-12-16T03:07:09.453ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
139872025-12-16T03:07:09.453ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
139882025-12-16T03:07:09.453ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-12-16 02:59:16.791641 UTC dns_group = internal generation = 1
139892025-12-16T03:07:09.453ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 278.67525ms iteration = 9 status = {"generation":1}
139902025-12-16T03:07:09.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
139912025-12-16T03:07:09.453ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
139922025-12-16T03:07:09.454ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1437, tv_nsec: 65280866 } })
139932025-12-16T03:07:09.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139942025-12-16T03:07:09.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139952025-12-16T03:07:09.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139962025-12-16T03:07:09.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139972025-12-16T03:07:09.468ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 195.861761ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
139982025-12-16T03:07:09.473ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139992025-12-16T03:07:09.473ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
140002025-12-16T03:07:09.473ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 200.466269ms iteration = 17 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
140012025-12-16T03:07:09.476ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
140022025-12-16T03:07:09.476ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140032025-12-16T03:07:09.476ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
140042025-12-16T03:07:09.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140052025-12-16T03:07:09.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140062025-12-16T03:07:09.479ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 206.943541ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
140072025-12-16T03:07:09.482ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
140082025-12-16T03:07:09.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140092025-12-16T03:07:09.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140102025-12-16T03:07:09.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140112025-12-16T03:07:09.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140122025-12-16T03:07:09.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140132025-12-16T03:07:09.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140142025-12-16T03:07:09.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140152025-12-16T03:07:09.494ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
140162025-12-16T03:07:09.494ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140172025-12-16T03:07:09.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140182025-12-16T03:07:09.496ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140192025-12-16T03:07:09.500ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
140202025-12-16T03:07:09.500ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140212025-12-16T03:07:09.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140222025-12-16T03:07:09.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140232025-12-16T03:07:09.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140242025-12-16T03:07:09.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140252025-12-16T03:07:09.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140262025-12-16T03:07:09.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140272025-12-16T03:07:09.514ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
140282025-12-16T03:07:09.514ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
140292025-12-16T03:07:09.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
140302025-12-16T03:07:09.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140312025-12-16T03:07:09.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140322025-12-16T03:07:09.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140332025-12-16T03:07:09.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140342025-12-16T03:07:09.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140352025-12-16T03:07:09.521ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 248.755411ms iteration = 17 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
140362025-12-16T03:07:09.523ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
140372025-12-16T03:07:09.524ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
140382025-12-16T03:07:09.524ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 307.911521ms iteration = 9 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
140392025-12-16T03:07:09.530ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ec7f925f-4c14-415c-9476-8dd4dc57451d step component = DeployNexusRecords step id = Ensure
140402025-12-16T03:07:09.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
140412025-12-16T03:07:09.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
140422025-12-16T03:07:09.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140432025-12-16T03:07:09.536ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
140442025-12-16T03:07:09.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
140452025-12-16T03:07:09.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
140462025-12-16T03:07:09.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
140472025-12-16T03:07:09.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
140482025-12-16T03:07:09.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140492025-12-16T03:07:09.550ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
140502025-12-16T03:07:09.566ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
140512025-12-16T03:07:09.566ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
140522025-12-16T03:07:09.567ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7098b678-08dd-48a5-a486-c6ed9243fe44", "content-length": "26", "date": "Tue, 16 Dec 2025 03:07:09 GMT"} })
140532025-12-16T03:07:09.567ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
140542025-12-16T03:07:09.569ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
140552025-12-16T03:07:09.573ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
140562025-12-16T03:07:09.573ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "cf1544fd-3e5b-4416-8d8a-2fa538347f27", "date": "Tue, 16 Dec 2025 03:07:09 GMT"} })
140572025-12-16T03:07:09.574ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
140582025-12-16T03:07:09.576ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
140592025-12-16T03:07:09.576ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 342.58578ms iteration = 9 status = {"errors":[],"probes_by_sled":{"b1c76b61-acee-4be6-b582-973c938d6b6e":0}}
140602025-12-16T03:07:09.580ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 307.950311ms iteration = 27 status = {"physical_disks_added":0}
140612025-12-16T03:07:09.596ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140622025-12-16T03:07:09.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140632025-12-16T03:07:09.599ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
140642025-12-16T03:07:09.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
140652025-12-16T03:07:09.602ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140662025-12-16T03:07:09.602ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140672025-12-16T03:07:09.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140682025-12-16T03:07:09.605ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140692025-12-16T03:07:09.605ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
140702025-12-16T03:07:09.605ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
140712025-12-16T03:07:09.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
140722025-12-16T03:07:09.606ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ec7f925f-4c14-415c-9476-8dd4dc57451d step component = SledAgent step id = Ensure
140732025-12-16T03:07:09.607ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140742025-12-16T03:07:09.608ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
140752025-12-16T03:07:09.608ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140762025-12-16T03:07:09.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140772025-12-16T03:07:09.610ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
140782025-12-16T03:07:09.610ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
140792025-12-16T03:07:09.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
140802025-12-16T03:07:09.611ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140812025-12-16T03:07:09.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140822025-12-16T03:07:09.613ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140832025-12-16T03:07:09.616ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
140842025-12-16T03:07:09.616ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140852025-12-16T03:07:09.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140862025-12-16T03:07:09.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140872025-12-16T03:07:09.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140882025-12-16T03:07:09.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140892025-12-16T03:07:09.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
140902025-12-16T03:07:09.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140912025-12-16T03:07:09.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140922025-12-16T03:07:09.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140932025-12-16T03:07:09.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140942025-12-16T03:07:09.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140952025-12-16T03:07:09.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
140962025-12-16T03:07:09.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140972025-12-16T03:07:09.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140982025-12-16T03:07:09.681ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
140992025-12-16T03:07:09.681ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
141002025-12-16T03:07:09.681ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 91d39877-6556-49b0-bd71-bfcdac177405 parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
141012025-12-16T03:07:09.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
141022025-12-16T03:07:09.682ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
141032025-12-16T03:07:09.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141042025-12-16T03:07:09.687ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
141052025-12-16T03:07:09.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
141062025-12-16T03:07:09.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
141072025-12-16T03:07:09.687ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
141082025-12-16T03:07:09.688ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "ff871e3c-3fca-4a18-846b-61a03a5e0caf", "date": "Tue, 16 Dec 2025 03:07:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
141092025-12-16T03:07:09.688ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
141102025-12-16T03:07:09.690ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
141112025-12-16T03:07:09.690ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "da29a1e2-f47e-426f-a0b4-f56406e622ef", "date": "Tue, 16 Dec 2025 03:07:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
141122025-12-16T03:07:09.690ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
141132025-12-16T03:07:09.690ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
141142025-12-16T03:07:09.690ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 475.215571ms iteration = 21 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
141152025-12-16T03:07:09.693ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
141162025-12-16T03:07:09.693ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 518.848103ms iteration = 11 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
141172025-12-16T03:07:09.743ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 568.708866ms iteration = 17 status = {}
141182025-12-16T03:07:09.743ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1437, tv_nsec: 354267328 } })
141192025-12-16T03:07:09.751ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
141202025-12-16T03:07:09.751ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
141212025-12-16T03:07:09.751ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "37ef8137-5852-46b6-8dfb-5cb7c610590e", "content-length": "372", "date": "Tue, 16 Dec 2025 03:07:09 GMT"} })
141222025-12-16T03:07:09.751ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
141232025-12-16T03:07:09.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
141242025-12-16T03:07:09.765ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
141252025-12-16T03:07:09.766ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
141262025-12-16T03:07:09.766ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
141272025-12-16T03:07:09.766ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "e785736a-716f-4c95-9c99-dbd7b95504b1", "date": "Tue, 16 Dec 2025 03:07:09 GMT"} })
141282025-12-16T03:07:09.766ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
141292025-12-16T03:07:09.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
141302025-12-16T03:07:09.769ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
141312025-12-16T03:07:09.769ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
141322025-12-16T03:07:09.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
141332025-12-16T03:07:09.770ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "b8ca8c61-96c0-4d58-8bdf-ea40604e9f4c", "date": "Tue, 16 Dec 2025 03:07:09 GMT"} }) sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
141342025-12-16T03:07:09.772ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
141352025-12-16T03:07:09.772ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ec7f925f-4c14-415c-9476-8dd4dc57451d step component = FirewallRules step id = Ensure
141362025-12-16T03:07:09.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
141372025-12-16T03:07:09.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
141382025-12-16T03:07:09.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
141392025-12-16T03:07:09.774ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
141402025-12-16T03:07:09.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
141412025-12-16T03:07:09.775ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
141422025-12-16T03:07:09.775ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
141432025-12-16T03:07:09.776ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
141442025-12-16T03:07:09.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
141452025-12-16T03:07:09.777ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "b27eeaf4-f188-417e-9b3e-7b8443c7ce0d", "date": "Tue, 16 Dec 2025 03:07:09 GMT"} })
141462025-12-16T03:07:09.778ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
141472025-12-16T03:07:09.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
141482025-12-16T03:07:09.779ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 506.120642ms iteration = 25 status = {}
141492025-12-16T03:07:09.779ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Dependency
141502025-12-16T03:07:09.779ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
141512025-12-16T03:07:09.779ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
141522025-12-16T03:07:09.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
141532025-12-16T03:07:09.780ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
141542025-12-16T03:07:09.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
141552025-12-16T03:07:09.780ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141562025-12-16T03:07:09.782ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
141572025-12-16T03:07:09.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
141582025-12-16T03:07:09.783ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
141592025-12-16T03:07:09.783ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
141602025-12-16T03:07:09.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
141612025-12-16T03:07:09.784ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141622025-12-16T03:07:09.784ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141632025-12-16T03:07:09.787ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
141642025-12-16T03:07:09.787ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
141652025-12-16T03:07:09.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
141662025-12-16T03:07:09.789ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
141672025-12-16T03:07:09.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
141682025-12-16T03:07:09.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
141692025-12-16T03:07:09.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
141702025-12-16T03:07:09.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
141712025-12-16T03:07:09.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
141722025-12-16T03:07:09.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
141732025-12-16T03:07:09.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141742025-12-16T03:07:09.801ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
141752025-12-16T03:07:09.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
141762025-12-16T03:07:09.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
141772025-12-16T03:07:09.804ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
141782025-12-16T03:07:09.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
141792025-12-16T03:07:09.806ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
141802025-12-16T03:07:09.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
141812025-12-16T03:07:09.807ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
141822025-12-16T03:07:09.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
141832025-12-16T03:07:09.809ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
141842025-12-16T03:07:09.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
141852025-12-16T03:07:09.811ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
141862025-12-16T03:07:09.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
141872025-12-16T03:07:09.813ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
141882025-12-16T03:07:09.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
141892025-12-16T03:07:09.814ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141902025-12-16T03:07:09.819ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
141912025-12-16T03:07:09.819ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
141922025-12-16T03:07:09.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
141932025-12-16T03:07:09.820ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
141942025-12-16T03:07:09.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
141952025-12-16T03:07:09.821ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
141962025-12-16T03:07:09.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
141972025-12-16T03:07:09.824ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
141982025-12-16T03:07:09.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
141992025-12-16T03:07:09.826ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142002025-12-16T03:07:09.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142012025-12-16T03:07:09.827ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142022025-12-16T03:07:09.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142032025-12-16T03:07:09.828ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142042025-12-16T03:07:09.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142052025-12-16T03:07:09.830ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142062025-12-16T03:07:09.830ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
142072025-12-16T03:07:09.835ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
142082025-12-16T03:07:09.835ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142092025-12-16T03:07:09.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142102025-12-16T03:07:09.835ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "fba62272-883f-4983-a71b-1bfacd99a507", "content-length": "691", "date": "Tue, 16 Dec 2025 03:07:09 GMT"} })
142112025-12-16T03:07:09.835ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
142122025-12-16T03:07:09.835ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
142132025-12-16T03:07:09.835ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
142142025-12-16T03:07:09.837ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142152025-12-16T03:07:09.837ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 564.34694ms iteration = 17 status = {}
142162025-12-16T03:07:09.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142172025-12-16T03:07:09.837ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1437, tv_nsec: 448588543 } })
142182025-12-16T03:07:09.838ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142192025-12-16T03:07:09.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142202025-12-16T03:07:09.840ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142212025-12-16T03:07:09.845ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
142222025-12-16T03:07:09.845ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142232025-12-16T03:07:09.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142242025-12-16T03:07:09.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142252025-12-16T03:07:09.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142262025-12-16T03:07:09.851ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
142272025-12-16T03:07:09.851ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142282025-12-16T03:07:09.851ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "e8e62b61-ef29-4af2-a719-9e5fd98fe8c8", "date": "Tue, 16 Dec 2025 03:07:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
142292025-12-16T03:07:09.851ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
142302025-12-16T03:07:09.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142312025-12-16T03:07:09.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142322025-12-16T03:07:09.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142332025-12-16T03:07:09.854ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "ca849a19-ee3a-4118-9fc8-653a75de9e12", "date": "Tue, 16 Dec 2025 03:07:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
142342025-12-16T03:07:09.854ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
142352025-12-16T03:07:09.856ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "02555557-872a-42f1-8489-363f361e1d3b", "date": "Tue, 16 Dec 2025 03:07:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
142362025-12-16T03:07:09.856ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142372025-12-16T03:07:09.856ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
142382025-12-16T03:07:09.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142392025-12-16T03:07:09.858ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
142402025-12-16T03:07:09.858ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 683.832923ms iteration = 11 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
142412025-12-16T03:07:09.858ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142422025-12-16T03:07:09.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142432025-12-16T03:07:09.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142442025-12-16T03:07:09.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142452025-12-16T03:07:09.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142462025-12-16T03:07:09.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142472025-12-16T03:07:09.863ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142482025-12-16T03:07:09.868ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
142492025-12-16T03:07:09.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142502025-12-16T03:07:09.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142512025-12-16T03:07:09.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142522025-12-16T03:07:09.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142532025-12-16T03:07:09.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142542025-12-16T03:07:09.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142552025-12-16T03:07:09.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142562025-12-16T03:07:09.877ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
142572025-12-16T03:07:09.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142582025-12-16T03:07:09.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142592025-12-16T03:07:09.877ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
142602025-12-16T03:07:09.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
142612025-12-16T03:07:09.880ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
142622025-12-16T03:07:09.880ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
142632025-12-16T03:07:09.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
142642025-12-16T03:07:09.883ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], Name("nexus"): [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
142652025-12-16T03:07:09.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142662025-12-16T03:07:09.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142672025-12-16T03:07:09.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142682025-12-16T03:07:09.887ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
142692025-12-16T03:07:09.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142702025-12-16T03:07:09.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142712025-12-16T03:07:09.888ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
142722025-12-16T03:07:09.888ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
142732025-12-16T03:07:09.888ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
142742025-12-16T03:07:09.896ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
142752025-12-16T03:07:09.896ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
142762025-12-16T03:07:09.897ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
142772025-12-16T03:07:09.897ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
142782025-12-16T03:07:09.897ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
142792025-12-16T03:07:09.897ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
142802025-12-16T03:07:09.897ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
142812025-12-16T03:07:09.897ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
142822025-12-16T03:07:09.897ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
142832025-12-16T03:07:09.897ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
142842025-12-16T03:07:09.898ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
142852025-12-16T03:07:09.898ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
142862025-12-16T03:07:09.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
142872025-12-16T03:07:09.901ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
142882025-12-16T03:07:09.901ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
142892025-12-16T03:07:09.908ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
142902025-12-16T03:07:09.908ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
142912025-12-16T03:07:09.909ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb7abd1a-4ef1-4d98-b5d4-0840fe10f7d0", "content-length": "372", "date": "Tue, 16 Dec 2025 03:07:09 GMT"} })
142922025-12-16T03:07:09.909ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
142932025-12-16T03:07:09.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
142942025-12-16T03:07:09.956ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), time_created: 2025-12-16T02:59:15.195809Z, time_modified: 2025-12-16T02:59:15.195809Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
142952025-12-16T03:07:09.956ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
142962025-12-16T03:07:09.973ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
142972025-12-16T03:07:09.973ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
142982025-12-16T03:07:09.973ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "261cb36f-3ce1-447a-bfee-fffe53e1cc55", "date": "Tue, 16 Dec 2025 03:07:09 GMT"} })
142992025-12-16T03:07:09.973ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
143002025-12-16T03:07:09.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
143012025-12-16T03:07:09.975ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
143022025-12-16T03:07:09.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
143032025-12-16T03:07:09.978ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
143042025-12-16T03:07:09.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
143052025-12-16T03:07:09.982ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
143062025-12-16T03:07:09.982ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
143072025-12-16T03:07:09.986ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "29864388-0665-47cf-93e2-6e34d158fbad", "date": "Tue, 16 Dec 2025 03:07:09 GMT"} })
143082025-12-16T03:07:09.987ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143092025-12-16T03:07:09.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143102025-12-16T03:07:09.989ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 210.204124ms iteration = 26 status = {}
143112025-12-16T03:07:09.990ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143122025-12-16T03:07:09.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143132025-12-16T03:07:09.991ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143142025-12-16T03:07:09.994ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
143152025-12-16T03:07:09.995ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
143162025-12-16T03:07:09.996ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "065d3890-8649-40dc-a328-68d8c78513ab", "date": "Tue, 16 Dec 2025 03:07:09 GMT"} })
143172025-12-16T03:07:10.000ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
143182025-12-16T03:07:10.000ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ec7f925f-4c14-415c-9476-8dd4dc57451d step component = Dns step id = Ensure
143192025-12-16T03:07:10.000ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
143202025-12-16T03:07:10.000ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143212025-12-16T03:07:10.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143222025-12-16T03:07:10.001ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143232025-12-16T03:07:10.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143242025-12-16T03:07:10.003ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143252025-12-16T03:07:10.007ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
143262025-12-16T03:07:10.008ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
143272025-12-16T03:07:10.008ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
143282025-12-16T03:07:10.008ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
143292025-12-16T03:07:10.008ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143302025-12-16T03:07:10.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143312025-12-16T03:07:10.010ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143322025-12-16T03:07:10.014ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
143332025-12-16T03:07:10.014ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143342025-12-16T03:07:10.014ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143352025-12-16T03:07:10.016ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
143362025-12-16T03:07:10.016ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
143372025-12-16T03:07:10.016ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143382025-12-16T03:07:10.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143392025-12-16T03:07:10.018ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143402025-12-16T03:07:10.022ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
143412025-12-16T03:07:10.022ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143422025-12-16T03:07:10.022ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143432025-12-16T03:07:10.024ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
143442025-12-16T03:07:10.024ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
143452025-12-16T03:07:10.024ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
143462025-12-16T03:07:10.024ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143472025-12-16T03:07:10.025ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143482025-12-16T03:07:10.025ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143492025-12-16T03:07:10.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143502025-12-16T03:07:10.027ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143512025-12-16T03:07:10.030ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
143522025-12-16T03:07:10.031ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "3e482978-9f8a-4ec5-909c-341f6e02334d", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
143532025-12-16T03:07:10.031ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
143542025-12-16T03:07:10.031ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
143552025-12-16T03:07:10.031ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "065fdee6-348c-4190-adef-cbbe4b1f8908", "content-length": "682", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
143562025-12-16T03:07:10.031ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
143572025-12-16T03:07:10.032ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
143582025-12-16T03:07:10.035ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
143592025-12-16T03:07:10.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
143602025-12-16T03:07:10.039ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
143612025-12-16T03:07:10.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
143622025-12-16T03:07:10.041ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
143632025-12-16T03:07:10.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
143642025-12-16T03:07:10.049ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 776.64892ms iteration = 17 status = {}
143652025-12-16T03:07:10.071ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
143662025-12-16T03:07:10.071ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
143672025-12-16T03:07:10.071ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
143682025-12-16T03:07:10.071ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143692025-12-16T03:07:10.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143702025-12-16T03:07:10.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143712025-12-16T03:07:10.077ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
143722025-12-16T03:07:10.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143732025-12-16T03:07:10.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143742025-12-16T03:07:10.079ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
143752025-12-16T03:07:10.079ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
143762025-12-16T03:07:10.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143772025-12-16T03:07:10.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143782025-12-16T03:07:10.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143792025-12-16T03:07:10.084ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
143802025-12-16T03:07:10.084ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143812025-12-16T03:07:10.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143822025-12-16T03:07:10.086ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
143832025-12-16T03:07:10.086ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
143842025-12-16T03:07:10.086ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143852025-12-16T03:07:10.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143862025-12-16T03:07:10.088ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143872025-12-16T03:07:10.090ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
143882025-12-16T03:07:10.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143892025-12-16T03:07:10.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143902025-12-16T03:07:10.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143912025-12-16T03:07:10.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143922025-12-16T03:07:10.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143932025-12-16T03:07:10.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143942025-12-16T03:07:10.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143952025-12-16T03:07:10.098ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
143962025-12-16T03:07:10.099ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
143972025-12-16T03:07:10.099ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
143982025-12-16T03:07:10.099ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ec7f925f-4c14-415c-9476-8dd4dc57451d step component = OmicronZones step id = Cleanup
143992025-12-16T03:07:10.099ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ec7f925f-4c14-415c-9476-8dd4dc57451d step component = OmicronZones step id = Cleanup
144002025-12-16T03:07:10.099ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ec7f925f-4c14-415c-9476-8dd4dc57451d step component = PhysicalDisks step id = Cleanup
144012025-12-16T03:07:10.099ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ec7f925f-4c14-415c-9476-8dd4dc57451d step component = Clickhouse step id = Ensure
144022025-12-16T03:07:10.099ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ec7f925f-4c14-415c-9476-8dd4dc57451d step component = Clickhouse step id = Ensure
144032025-12-16T03:07:10.129ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
144042025-12-16T03:07:10.134ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "906ade16-efe3-476b-9a98-7ff2c4ddfb60", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
144052025-12-16T03:07:10.136ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ec7f925f-4c14-415c-9476-8dd4dc57451d step component = SupportBundles step id = Cleanup
144062025-12-16T03:07:10.136ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
144072025-12-16T03:07:10.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
144082025-12-16T03:07:10.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144092025-12-16T03:07:10.139ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
144102025-12-16T03:07:10.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
144112025-12-16T03:07:10.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
144122025-12-16T03:07:10.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
144132025-12-16T03:07:10.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
144142025-12-16T03:07:10.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144152025-12-16T03:07:10.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
144162025-12-16T03:07:10.185ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ec7f925f-4c14-415c-9476-8dd4dc57451d step component = OmicronZones step id = Cleanup
144172025-12-16T03:07:10.185ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
144182025-12-16T03:07:10.185ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
144192025-12-16T03:07:10.185ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
144202025-12-16T03:07:10.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
144212025-12-16T03:07:10.187ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144222025-12-16T03:07:10.188ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
144232025-12-16T03:07:10.188ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
144242025-12-16T03:07:10.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
144252025-12-16T03:07:10.232ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
144262025-12-16T03:07:10.232ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(a91734db-9ac7-4793-9c32-ddbb21f8d986 (blueprint), false)
144272025-12-16T03:07:10.232ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ec7f925f-4c14-415c-9476-8dd4dc57451d step component = Cockroach step id = Ensure
144282025-12-16T03:07:10.232ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
144292025-12-16T03:07:10.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
144302025-12-16T03:07:10.251ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
144312025-12-16T03:07:10.251ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ec7f925f-4c14-415c-9476-8dd4dc57451d step component = MgsUpdates step id = Ensure
144322025-12-16T03:07:10.251ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 26 reason = Dependency
144332025-12-16T03:07:10.251ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 10 reason = Dependency
144342025-12-16T03:07:10.251ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
144352025-12-16T03:07:10.251ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
144362025-12-16T03:07:10.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
144372025-12-16T03:07:10.252ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.077268199s iteration = 10 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"9db36a69-617b-4173-a62c-91a80b7002a2","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"9db36a69-617b-4173-a62c-91a80b7002a2","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":2900,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":65700513,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":65700463,"secs":0}},"event_index":1,"execution_id":"9db36a69-617b-4173-a62c-91a80b7002a2","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":65712243,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":289809082,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":289809042,"secs":0}},"event_index":2,"execution_id":"9db36a69-617b-4173-a62c-91a80b7002a2","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":355541714,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":76726542,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":76726512,"secs":0}},"event_index":3,"execution_id":"9db36a69-617b-4173-a62c-91a80b7002a2","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":432281947,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":165514867,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":165514837,"secs":0}},"event_index":4,"execution_id":"9db36a69-617b-4173-a62c-91a80b7002a2","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":597809514,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":227741416,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":227741376,"secs":0}},"event_index":5,"execution_id":"9db36a69-617b-4173-a62c-91a80b7002a2","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":825563640,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":99012535,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":99012495,"secs":0}},"event_index":6,"execution_id":"9db36a69-617b-4173-a62c-91a80b7002a2","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":924586315,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3930,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":3890,"secs":0}},"event_index":7,"execution_id":"9db36a69-617b-4173-a62c-91a80b7002a2","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":924602015,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2300,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2260,"secs":0}},"event_index":8,"execution_id":"9db36a69-617b-4173-a62c-91a80b7002a2","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":924606435,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3110,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3080,"secs":0}},"event_index":9,"execution_id":"9db36a69-617b-4173-a62c-91a80b7002a2","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":924611945,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1720,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1680,"secs":0}},"event_index":10,"execution_id":"9db36a69-617b-4173-a62c-91a80b7002a2","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":924615185,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":37020909,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":37020869,"secs":0}},"event_index":11,"execution_id":"9db36a69-617b-4173-a62c-91a80b7002a2","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":961638064,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":49200014,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":49199974,"secs":0}},"event_index":12,"execution_id":"9db36a69-617b-4173-a62c-91a80b7002a2","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":10850768,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":46993045,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":46993005,"secs":0}},"event_index":13,"execution_id":"9db36a69-617b-4173-a62c-91a80b7002a2","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":57866553,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":19053511,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":19053471,"secs":0}},"event_index":14,"execution_id":"9db36a69-617b-4173-a62c-91a80b7002a2","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":76935625,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5340,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5300,"secs":0}},"event_index":15,"execution_id":"9db36a69-617b-4173-a62c-91a80b7002a2","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":76953125,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986"}
144382025-12-16T03:07:10.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
144392025-12-16T03:07:10.253ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144402025-12-16T03:07:10.253ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144412025-12-16T03:07:10.256ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
144422025-12-16T03:07:10.256ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
144432025-12-16T03:07:10.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
144442025-12-16T03:07:10.257ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
144452025-12-16T03:07:10.257ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
144462025-12-16T03:07:10.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
144472025-12-16T03:07:10.295ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
144482025-12-16T03:07:10.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
144492025-12-16T03:07:10.296ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144502025-12-16T03:07:10.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
144512025-12-16T03:07:10.299ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
144522025-12-16T03:07:10.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
144532025-12-16T03:07:10.301ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
144542025-12-16T03:07:10.301ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 49.431777ms iteration = 26 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
144552025-12-16T03:07:10.306ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(b6c51ddf-9f27-432f-9d24-9fbc42795d0c, 0), (e20dce38-fb3a-4fef-ab59-917046b281c6, 0), (d36b7584-5102-4a4f-888b-fa1c6fea6614, 0), (b97ed3f8-6719-4b04-8f47-be78e07de798, 0)] collection_id = b6c51ddf-9f27-432f-9d24-9fbc42795d0c
144562025-12-16T03:07:10.306ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
144572025-12-16T03:07:10.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
144582025-12-16T03:07:10.444ZWARNnexus (datastore): Retryable transaction failure file = nexus/db-queries/src/transaction_retry.rs:165 retry_after (ms) = 0 transaction = inventory_delete_collection
144592025-12-16T03:07:10.478ZINFOnexus (datastore): transaction completed attempts = 2 file = nexus/db-queries/src/transaction_retry.rs:121 transaction = inventory_delete_collection
144602025-12-16T03:07:10.478ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = b6c51ddf-9f27-432f-9d24-9fbc42795d0c file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
144612025-12-16T03:07:10.478ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
144622025-12-16T03:07:10.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
144632025-12-16T03:07:10.481ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(e20dce38-fb3a-4fef-ab59-917046b281c6, 0), (d36b7584-5102-4a4f-888b-fa1c6fea6614, 0), (b97ed3f8-6719-4b04-8f47-be78e07de798, 0)]
144642025-12-16T03:07:10.481ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1438, tv_nsec: 92577492 } })
144652025-12-16T03:07:10.512ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("20488683-5778-4090-b36e-e6f0d80a682b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("35dbaad1-8536-44a0-8a0a-35d7c39dcfbb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b57fa226-2428-4a54-b85d-60409e28b484.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("d812a3e2-48ae-4812-a1cc-7eb7623e4e00.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::5)) }], valid_until: Instant { tv_sec: 1438, tv_nsec: 123428443 } })
144662025-12-16T03:07:10.513ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
144672025-12-16T03:07:10.513ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
144682025-12-16T03:07:10.513ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
144692025-12-16T03:07:10.513ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6de45bd-abc9-4323-a1f6-b0c3211f2ed0", "content-length": "476", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
144702025-12-16T03:07:10.513ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
144712025-12-16T03:07:10.514ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3828b89-bb12-4a2d-842b-c94ecd8d6ec7", "content-length": "734", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
144722025-12-16T03:07:10.514ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
144732025-12-16T03:07:10.514ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e064fc62-b1fe-4f6e-8cbe-3a60cfa08d1d", "content-length": "111", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
144742025-12-16T03:07:10.514ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
144752025-12-16T03:07:10.515ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8e1ebe6-9845-49dc-accf-594a352d9aa3", "content-length": "111", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
144762025-12-16T03:07:10.515ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
144772025-12-16T03:07:10.515ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fb27a6c-42b6-4a32-aacc-1274b49f313a", "content-length": "163", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
144782025-12-16T03:07:10.515ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
144792025-12-16T03:07:10.515ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "139b00c2-0720-4efb-b9bf-f86cf166120f", "content-length": "163", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
144802025-12-16T03:07:10.515ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
144812025-12-16T03:07:10.516ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc7c75c3-cf5d-4958-9a02-1242cfb99838", "content-length": "166", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
144822025-12-16T03:07:10.516ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
144832025-12-16T03:07:10.516ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "45f658ff-83ab-413c-91c9-5c77d81e69f6", "content-length": "166", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
144842025-12-16T03:07:10.516ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
144852025-12-16T03:07:10.516ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "095cad56-d3f8-448e-a447-5714a61d4293", "content-length": "702", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
144862025-12-16T03:07:10.516ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
144872025-12-16T03:07:10.517ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "604efe01-35fe-47ae-abd6-6b3f18e32bf1", "content-length": "727", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
144882025-12-16T03:07:10.517ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
144892025-12-16T03:07:10.517ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c70b4cc-abfa-4302-a3b7-294affa1c2fc", "content-length": "729", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
144902025-12-16T03:07:10.517ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
144912025-12-16T03:07:10.517ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d5c05b2-6e32-4cec-9a0b-f72e01fea8cf", "content-length": "728", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
144922025-12-16T03:07:10.517ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
144932025-12-16T03:07:10.517ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "366b0183-7968-417e-8d30-15693129dfd3", "content-length": "722", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
144942025-12-16T03:07:10.517ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
144952025-12-16T03:07:10.517ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "4292abe5-ba8e-4b83-9207-b85e342383a2", "content-length": "10", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
144962025-12-16T03:07:10.517ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
144972025-12-16T03:07:10.518ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7462a74-6a47-4f24-811a-6ba9830d2d1b", "content-length": "144", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
144982025-12-16T03:07:10.518ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
144992025-12-16T03:07:10.518ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0af756a-776c-4a6b-bfa3-b98f8c369971", "content-length": "147", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145002025-12-16T03:07:10.518ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
145012025-12-16T03:07:10.519ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "994279a6-69cb-4bb8-a042-7c0057e627e9", "content-length": "109", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145022025-12-16T03:07:10.519ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
145032025-12-16T03:07:10.519ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5b1df2a-1084-4e63-afd6-0cba85c07e2d", "content-length": "109", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145042025-12-16T03:07:10.519ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
145052025-12-16T03:07:10.520ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b38e631-cd00-43d8-a0db-431c7d832426", "content-length": "163", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145062025-12-16T03:07:10.520ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
145072025-12-16T03:07:10.521ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "929c826a-57fb-4564-ac12-64c85f392612", "content-length": "163", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145082025-12-16T03:07:10.521ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
145092025-12-16T03:07:10.521ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ce1fea5-10e4-485a-96e0-9b9bafb4075b", "content-length": "166", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145102025-12-16T03:07:10.521ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
145112025-12-16T03:07:10.521ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5182273-e46b-4b45-89fb-3862d7a24895", "content-length": "166", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145122025-12-16T03:07:10.521ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
145132025-12-16T03:07:10.522ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a4fcb07-0b2f-4a8e-8ac7-b4b8f185dc0c", "content-length": "702", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145142025-12-16T03:07:10.522ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
145152025-12-16T03:07:10.522ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "53eecf9b-f3bb-4ad0-acc9-8d2439494957", "content-length": "727", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145162025-12-16T03:07:10.522ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
145172025-12-16T03:07:10.522ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "81f808c4-f196-4c77-b4ca-6842b4b9ca90", "content-length": "729", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145182025-12-16T03:07:10.522ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
145192025-12-16T03:07:10.523ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "242b873f-4653-46b6-9652-f0928f9568c3", "content-length": "728", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145202025-12-16T03:07:10.523ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
145212025-12-16T03:07:10.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
145222025-12-16T03:07:10.525ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145232025-12-16T03:07:10.528ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
145242025-12-16T03:07:10.528ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
145252025-12-16T03:07:10.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
145262025-12-16T03:07:10.530ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145272025-12-16T03:07:10.532ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
145282025-12-16T03:07:10.532ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
145292025-12-16T03:07:10.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
145302025-12-16T03:07:10.567ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
145312025-12-16T03:07:10.567ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
145322025-12-16T03:07:10.619ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "eeab55c3-da2c-4f93-bfe4-62a4ee218818", "content-length": "95612", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145332025-12-16T03:07:10.621ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
145342025-12-16T03:07:10.621ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
145352025-12-16T03:07:10.621ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
145362025-12-16T03:07:10.621ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
145372025-12-16T03:07:10.621ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
145382025-12-16T03:07:10.621ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
145392025-12-16T03:07:10.621ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
145402025-12-16T03:07:10.621ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
145412025-12-16T03:07:10.621ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1634f70f-c185-404f-ba0e-780a2dac7b92", "content-length": "64", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145422025-12-16T03:07:10.621ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
145432025-12-16T03:07:10.621ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7cf92d7-a6bf-4f73-bd89-a0888fa4190c", "content-length": "64", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145442025-12-16T03:07:10.621ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f4f34fc-971b-4a48-9686-f70684d22f08", "content-length": "64", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145452025-12-16T03:07:10.621ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e32de42-0e63-4b19-9cad-ef11d4c2c1e3", "content-length": "64", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145462025-12-16T03:07:10.621ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0241963-805b-4542-a237-83a7652298e4", "content-length": "64", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145472025-12-16T03:07:10.621ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
145482025-12-16T03:07:10.621ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
145492025-12-16T03:07:10.621ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
145502025-12-16T03:07:10.621ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
145512025-12-16T03:07:10.665ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ad5d398-1a90-4259-9cd0-7f78bd2f5ca3", "content-length": "401890", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145522025-12-16T03:07:10.667ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf7d2bbd-7b06-43bf-90a7-11e5cec92904", "content-length": "403645", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145532025-12-16T03:07:10.668ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "60244c1d-f9a1-44f9-bc66-d5898407f496", "content-length": "404373", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145542025-12-16T03:07:10.669ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "39d742ac-d654-4eb3-a38b-bbaf67eb8a2c", "content-length": "434197", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145552025-12-16T03:07:10.672ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "63a5f116-5892-48c5-846f-cebd7512f89b", "content-length": "405896", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145562025-12-16T03:07:10.674ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
145572025-12-16T03:07:10.674ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
145582025-12-16T03:07:10.674ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
145592025-12-16T03:07:10.674ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
145602025-12-16T03:07:10.674ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
145612025-12-16T03:07:10.705ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c (omicron_zone)
145622025-12-16T03:07:10.705ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
145632025-12-16T03:07:10.708ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "9379a5e1-b2e1-4d84-a5ec-0be75b8e2166", "content-length": "125", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145642025-12-16T03:07:10.709ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
145652025-12-16T03:07:10.803ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
145662025-12-16T03:07:10.803ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
145672025-12-16T03:07:10.803ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d60e6ac-8463-43bb-909b-af770962d756", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145682025-12-16T03:07:10.803ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
145692025-12-16T03:07:10.805ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
145702025-12-16T03:07:10.805ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
145712025-12-16T03:07:10.806ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5c9ab91-181d-4a6e-9c85-b240caf01d32", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145722025-12-16T03:07:10.806ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
145732025-12-16T03:07:10.808ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
145742025-12-16T03:07:10.808ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
145752025-12-16T03:07:10.808ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "83cdf54a-99a3-44e8-9b5b-3738b6707c3a", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:07:10 GMT"} })
145762025-12-16T03:07:10.808ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
145772025-12-16T03:07:10.809ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
145782025-12-16T03:07:10.809ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
145792025-12-16T03:07:10.811ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
145802025-12-16T03:07:10.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
145812025-12-16T03:07:10.813ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145822025-12-16T03:07:10.815ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
145832025-12-16T03:07:10.815ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
145842025-12-16T03:07:10.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
145852025-12-16T03:07:11.025ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 40cdf0c1-4d12-4355-9a49-17ccd3837b4c file = nexus/db-queries/src/db/datastore/inventory.rs:1613
145862025-12-16T03:07:11.025ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 40cdf0c1-4d12-4355-9a49-17ccd3837b4c time_started = 2025-12-16 03:07:10.513292 UTC
145872025-12-16T03:07:11.025ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 43 reason = Dependency
145882025-12-16T03:07:11.025ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
145892025-12-16T03:07:11.025ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 29 reason = Dependency
145902025-12-16T03:07:11.025ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
145912025-12-16T03:07:11.025ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 21.96\u{b5}s iteration = 29 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
145922025-12-16T03:07:11.025ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 773.638412ms iteration = 10 status = {"collection_id":"40cdf0c1-4d12-4355-9a49-17ccd3837b4c","time_done":"2025-12-16 03:07:10.809615 UTC","time_started":"2025-12-16 03:07:10.513292 UTC"}
145932025-12-16T03:07:11.025ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
145942025-12-16T03:07:11.027ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145952025-12-16T03:07:11.030ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
145962025-12-16T03:07:11.030ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
145972025-12-16T03:07:11.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
145982025-12-16T03:07:11.066ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 286 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 9d0198a2-70fd-4044-943e-956cf5a8b1ea response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
145992025-12-16T03:07:11.084ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
146002025-12-16T03:07:11.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
146012025-12-16T03:07:11.129ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 103.916966ms iteration = 43 status = {"Loaded":{"collection_id":"40cdf0c1-4d12-4355-9a49-17ccd3837b4c","time_loaded":"2025-12-16T03:07:11.025270227Z","time_started":"2025-12-16T03:07:10.513292Z"}}
146022025-12-16T03:07:11.129ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 13 reason = Dependency
146032025-12-16T03:07:11.129ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Dependency
146042025-12-16T03:07:11.129ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 30 reason = Dependency
146052025-12-16T03:07:11.129ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 22 reason = Dependency
146062025-12-16T03:07:11.129ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
146072025-12-16T03:07:11.129ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 8.661\u{b5}s iteration = 30 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
146082025-12-16T03:07:11.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146092025-12-16T03:07:11.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
146102025-12-16T03:07:11.129ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
146112025-12-16T03:07:11.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
146122025-12-16T03:07:11.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146132025-12-16T03:07:11.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
146142025-12-16T03:07:11.131ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146152025-12-16T03:07:11.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146162025-12-16T03:07:11.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146172025-12-16T03:07:11.134ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
146182025-12-16T03:07:11.134ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
146192025-12-16T03:07:11.134ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
146202025-12-16T03:07:11.134ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
146212025-12-16T03:07:11.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
146222025-12-16T03:07:11.134ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
146232025-12-16T03:07:11.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
146242025-12-16T03:07:11.134ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146252025-12-16T03:07:11.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146262025-12-16T03:07:11.136ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146272025-12-16T03:07:11.136ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146282025-12-16T03:07:11.136ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
146292025-12-16T03:07:11.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
146302025-12-16T03:07:11.138ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146312025-12-16T03:07:11.138ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
146322025-12-16T03:07:11.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
146332025-12-16T03:07:11.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
146342025-12-16T03:07:11.139ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
146352025-12-16T03:07:11.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146362025-12-16T03:07:11.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146372025-12-16T03:07:11.140ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
146382025-12-16T03:07:11.140ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
146392025-12-16T03:07:11.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
146402025-12-16T03:07:11.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
146412025-12-16T03:07:11.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
146422025-12-16T03:07:11.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146432025-12-16T03:07:11.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146442025-12-16T03:07:11.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146452025-12-16T03:07:11.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146462025-12-16T03:07:11.149ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
146472025-12-16T03:07:11.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146482025-12-16T03:07:11.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146492025-12-16T03:07:11.149ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
146502025-12-16T03:07:11.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
146512025-12-16T03:07:11.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
146522025-12-16T03:07:11.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146532025-12-16T03:07:11.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146542025-12-16T03:07:11.153ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
146552025-12-16T03:07:11.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146562025-12-16T03:07:11.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146572025-12-16T03:07:11.154ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
146582025-12-16T03:07:11.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
146592025-12-16T03:07:11.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
146602025-12-16T03:07:11.156ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
146612025-12-16T03:07:11.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
146622025-12-16T03:07:11.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
146632025-12-16T03:07:11.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146642025-12-16T03:07:11.159ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
146652025-12-16T03:07:11.159ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
146662025-12-16T03:07:11.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
146672025-12-16T03:07:11.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146682025-12-16T03:07:11.163ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
146692025-12-16T03:07:11.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
146702025-12-16T03:07:11.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
146712025-12-16T03:07:11.165ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
146722025-12-16T03:07:11.165ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 35.957266ms iteration = 13 status = {"blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","inventory_collection_id":"40cdf0c1-4d12-4355-9a49-17ccd3837b4c","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
146732025-12-16T03:07:11.194ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 64.787441ms iteration = 28 status = {"physical_disks_added":0}
146742025-12-16T03:07:11.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146752025-12-16T03:07:11.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146762025-12-16T03:07:11.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146772025-12-16T03:07:11.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146782025-12-16T03:07:11.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146792025-12-16T03:07:11.206ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
146802025-12-16T03:07:11.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146812025-12-16T03:07:11.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146822025-12-16T03:07:11.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146832025-12-16T03:07:11.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146842025-12-16T03:07:11.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146852025-12-16T03:07:11.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
146862025-12-16T03:07:11.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146872025-12-16T03:07:11.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146882025-12-16T03:07:11.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146892025-12-16T03:07:11.218ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
146902025-12-16T03:07:11.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146912025-12-16T03:07:11.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146922025-12-16T03:07:11.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146932025-12-16T03:07:11.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146942025-12-16T03:07:11.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146952025-12-16T03:07:11.225ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
146962025-12-16T03:07:11.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146972025-12-16T03:07:11.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
146982025-12-16T03:07:11.227ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
146992025-12-16T03:07:11.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147002025-12-16T03:07:11.229ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147012025-12-16T03:07:11.232ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
147022025-12-16T03:07:11.232ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147032025-12-16T03:07:11.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147042025-12-16T03:07:11.234ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147052025-12-16T03:07:11.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147062025-12-16T03:07:11.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147072025-12-16T03:07:11.238ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
147082025-12-16T03:07:11.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147092025-12-16T03:07:11.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147102025-12-16T03:07:11.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147112025-12-16T03:07:11.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147122025-12-16T03:07:11.256ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147132025-12-16T03:07:11.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147142025-12-16T03:07:11.258ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147152025-12-16T03:07:11.261ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
147162025-12-16T03:07:11.261ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147172025-12-16T03:07:11.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147182025-12-16T03:07:11.262ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147192025-12-16T03:07:11.265ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
147202025-12-16T03:07:11.266ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147212025-12-16T03:07:11.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147222025-12-16T03:07:11.268ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147232025-12-16T03:07:11.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147242025-12-16T03:07:11.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147252025-12-16T03:07:11.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147262025-12-16T03:07:11.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147272025-12-16T03:07:11.274ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
147282025-12-16T03:07:11.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147292025-12-16T03:07:11.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147302025-12-16T03:07:11.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147312025-12-16T03:07:11.279ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
147322025-12-16T03:07:11.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147332025-12-16T03:07:11.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147342025-12-16T03:07:11.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147352025-12-16T03:07:11.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147362025-12-16T03:07:11.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147372025-12-16T03:07:11.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147382025-12-16T03:07:11.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147392025-12-16T03:07:11.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
147402025-12-16T03:07:11.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147412025-12-16T03:07:11.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147422025-12-16T03:07:11.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147432025-12-16T03:07:11.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147442025-12-16T03:07:11.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147452025-12-16T03:07:11.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
147462025-12-16T03:07:11.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147472025-12-16T03:07:11.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147482025-12-16T03:07:11.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147492025-12-16T03:07:11.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147502025-12-16T03:07:11.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147512025-12-16T03:07:11.300ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
147522025-12-16T03:07:11.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147532025-12-16T03:07:11.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147542025-12-16T03:07:11.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147552025-12-16T03:07:11.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147562025-12-16T03:07:11.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147572025-12-16T03:07:11.306ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
147582025-12-16T03:07:11.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147592025-12-16T03:07:11.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147602025-12-16T03:07:11.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147612025-12-16T03:07:11.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147622025-12-16T03:07:11.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147632025-12-16T03:07:11.312ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
147642025-12-16T03:07:11.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147652025-12-16T03:07:11.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147662025-12-16T03:07:11.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147672025-12-16T03:07:11.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147682025-12-16T03:07:11.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147692025-12-16T03:07:11.317ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
147702025-12-16T03:07:11.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147712025-12-16T03:07:11.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147722025-12-16T03:07:11.356ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
147732025-12-16T03:07:11.356ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
147742025-12-16T03:07:11.356ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 8708992e-dfc6-416b-8913-bb57ab5559c7 parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
147752025-12-16T03:07:11.356ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147762025-12-16T03:07:11.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147772025-12-16T03:07:11.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147782025-12-16T03:07:11.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
147792025-12-16T03:07:11.360ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147802025-12-16T03:07:11.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147812025-12-16T03:07:11.363ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
147822025-12-16T03:07:11.363ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
147832025-12-16T03:07:11.363ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 234.39459ms iteration = 22 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
147842025-12-16T03:07:14.272ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 98 reason = Timeout
147852025-12-16T03:07:14.272ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
147862025-12-16T03:07:14.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
147872025-12-16T03:07:14.274ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147882025-12-16T03:07:14.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
147892025-12-16T03:07:14.277ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
147902025-12-16T03:07:14.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
147912025-12-16T03:07:14.278ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
147922025-12-16T03:07:14.278ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.380731ms iteration = 98 status = {"config_updated":false}
147932025-12-16T03:07:19.272ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 50 reason = Timeout
147942025-12-16T03:07:19.272ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
147952025-12-16T03:07:19.272ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
147962025-12-16T03:07:19.272ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
147972025-12-16T03:07:19.272ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 99 reason = Timeout
147982025-12-16T03:07:19.272ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
147992025-12-16T03:07:19.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
148002025-12-16T03:07:19.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
148012025-12-16T03:07:19.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
148022025-12-16T03:07:19.274ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
148032025-12-16T03:07:19.275ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148042025-12-16T03:07:19.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
148052025-12-16T03:07:19.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148062025-12-16T03:07:19.277ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.73039ms iteration = 50 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
148072025-12-16T03:07:19.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
148082025-12-16T03:07:19.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
148092025-12-16T03:07:19.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
148102025-12-16T03:07:19.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
148112025-12-16T03:07:19.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
148122025-12-16T03:07:19.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
148132025-12-16T03:07:19.280ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
148142025-12-16T03:07:19.280ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.116383ms iteration = 99 status = {"config_updated":false}
148152025-12-16T03:07:19.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
148162025-12-16T03:07:19.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
148172025-12-16T03:07:19.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148182025-12-16T03:07:19.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
148192025-12-16T03:07:19.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
148202025-12-16T03:07:19.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
148212025-12-16T03:07:19.297ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
148222025-12-16T03:07:19.297ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.887455ms iteration = 50 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
148232025-12-16T03:07:21.066ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 283 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = f26d8b4e-c766-40dc-a25b-7b82cf2d5be0 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
148242025-12-16T03:07:24.272ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 44 reason = Timeout
148252025-12-16T03:07:24.272ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 34 reason = Timeout
148262025-12-16T03:07:24.272ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
148272025-12-16T03:07:24.272ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 34 reason = Timeout
148282025-12-16T03:07:24.272ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
148292025-12-16T03:07:24.272ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
148302025-12-16T03:07:24.272ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 100 reason = Timeout
148312025-12-16T03:07:24.272ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
148322025-12-16T03:07:24.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
148332025-12-16T03:07:24.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
148342025-12-16T03:07:24.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
148352025-12-16T03:07:24.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
148362025-12-16T03:07:24.273ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
148372025-12-16T03:07:24.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
148382025-12-16T03:07:24.273ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148392025-12-16T03:07:24.274ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148402025-12-16T03:07:24.274ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148412025-12-16T03:07:24.275ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148422025-12-16T03:07:24.277ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
148432025-12-16T03:07:24.277ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
148442025-12-16T03:07:24.277ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
148452025-12-16T03:07:24.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
148462025-12-16T03:07:24.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
148472025-12-16T03:07:24.278ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
148482025-12-16T03:07:24.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
148492025-12-16T03:07:24.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
148502025-12-16T03:07:24.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
148512025-12-16T03:07:24.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
148522025-12-16T03:07:24.279ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 7.708357ms iteration = 34 status = "NoSitrep"
148532025-12-16T03:07:24.280ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 40cdf0c1-4d12-4355-9a49-17ccd3837b4c original_time_started = 2025-12-16 03:07:10.513292 UTC
148542025-12-16T03:07:24.280ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.349686ms iteration = 44 status = {"Loaded":{"collection_id":"40cdf0c1-4d12-4355-9a49-17ccd3837b4c","time_loaded":"2025-12-16T03:07:24.272001031Z","time_started":"2025-12-16T03:07:10.513292Z"}}
148552025-12-16T03:07:24.280ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
148562025-12-16T03:07:24.280ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.351246ms iteration = 100 status = {"config_updated":false}
148572025-12-16T03:07:24.280ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
148582025-12-16T03:07:24.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
148592025-12-16T03:07:24.281ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
148602025-12-16T03:07:24.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
148612025-12-16T03:07:24.282ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148622025-12-16T03:07:24.284ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
148632025-12-16T03:07:24.284ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
148642025-12-16T03:07:24.284ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 12.747711ms iteration = 34 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
148652025-12-16T03:07:29.272ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 51 reason = Timeout
148662025-12-16T03:07:29.273ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
148672025-12-16T03:07:29.273ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
148682025-12-16T03:07:29.273ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 101 reason = Timeout
148692025-12-16T03:07:29.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
148702025-12-16T03:07:29.273ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
148712025-12-16T03:07:29.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
148722025-12-16T03:07:29.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
148732025-12-16T03:07:29.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
148742025-12-16T03:07:29.275ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
148752025-12-16T03:07:29.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148762025-12-16T03:07:29.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
148772025-12-16T03:07:29.275ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148782025-12-16T03:07:29.277ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.422606ms iteration = 51 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
148792025-12-16T03:07:29.279ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
148802025-12-16T03:07:29.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
148812025-12-16T03:07:29.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
148822025-12-16T03:07:29.279ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
148832025-12-16T03:07:29.279ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
148842025-12-16T03:07:29.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
148852025-12-16T03:07:29.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
148862025-12-16T03:07:29.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
148872025-12-16T03:07:29.282ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
148882025-12-16T03:07:29.282ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 9.310978ms iteration = 101 status = {"config_updated":false}
148892025-12-16T03:07:29.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148902025-12-16T03:07:29.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
148912025-12-16T03:07:29.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
148922025-12-16T03:07:29.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
148932025-12-16T03:07:29.298ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
148942025-12-16T03:07:29.298ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.713796ms iteration = 51 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
148952025-12-16T03:07:31.067ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 498 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 339e00b2-5b0f-4954-b5ac-b25564229e80 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
148962025-12-16T03:07:34.272ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 102 reason = Timeout
148972025-12-16T03:07:34.272ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
148982025-12-16T03:07:34.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
148992025-12-16T03:07:34.275ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149002025-12-16T03:07:34.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
149012025-12-16T03:07:34.278ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
149022025-12-16T03:07:34.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
149032025-12-16T03:07:34.280ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
149042025-12-16T03:07:34.280ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.112073ms iteration = 102 status = {"config_updated":false}
149052025-12-16T03:07:36.060ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 19a6dad9-9968-4ea1-8a0c-d6c571aee901 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending version = 1.0.0
149062025-12-16T03:07:36.060ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 19a6dad9-9968-4ea1-8a0c-d6c571aee901 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
149072025-12-16T03:07:36.060ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 19a6dad9-9968-4ea1-8a0c-d6c571aee901 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
149082025-12-16T03:07:36.062ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 2649 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 19a6dad9-9968-4ea1-8a0c-d6c571aee901 response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending
149092025-12-16T03:07:36.063ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 755639ff-d93c-4fc9-bf46-6f9f48d7c346 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D version = 1.0.0
149102025-12-16T03:07:36.063ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 755639ff-d93c-4fc9-bf46-6f9f48d7c346 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
149112025-12-16T03:07:36.063ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 755639ff-d93c-4fc9-bf46-6f9f48d7c346 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
149122025-12-16T03:07:36.065ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 2004 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59290 req_id = 755639ff-d93c-4fc9-bf46-6f9f48d7c346 response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D
149132025-12-16T03:07:39.174ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
149142025-12-16T03:07:39.174ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 18 reason = Timeout
149152025-12-16T03:07:39.174ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
149162025-12-16T03:07:39.174ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
149172025-12-16T03:07:39.174ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
149182025-12-16T03:07:39.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
149192025-12-16T03:07:39.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
149202025-12-16T03:07:39.176ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1466, tv_nsec: 787251534 } })
149212025-12-16T03:07:39.176ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1466, tv_nsec: 787259255 } })
149222025-12-16T03:07:39.177ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
149232025-12-16T03:07:39.177ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1466, tv_nsec: 787906693 } })
149242025-12-16T03:07:39.178ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.623008ms iteration = 18 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
149252025-12-16T03:07:39.251ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
149262025-12-16T03:07:39.252ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 78.433994ms iteration = 18 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
149272025-12-16T03:07:39.264ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
149282025-12-16T03:07:39.264ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
149292025-12-16T03:07:39.264ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f284b65e-db8c-446f-a76e-f89ef6375546", "content-length": "26", "date": "Tue, 16 Dec 2025 03:07:39 GMT"} })
149302025-12-16T03:07:39.264ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
149312025-12-16T03:07:39.266ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
149322025-12-16T03:07:39.267ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1466, tv_nsec: 877748521 } })
149332025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 35 reason = Timeout
149342025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
149352025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
149362025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
149372025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 45 reason = Timeout
149382025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
149392025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
149402025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
149412025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 35 reason = Timeout
149422025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 52 reason = Timeout
149432025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 27 reason = Timeout
149442025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
149452025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
149462025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 18 reason = Timeout
149472025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
149482025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
149492025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
149502025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
149512025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 18 reason = Timeout
149522025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
149532025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
149542025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 103 reason = Timeout
149552025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
149562025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 18 reason = Timeout
149572025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 18 reason = Timeout
149582025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
149592025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 13.09\u{b5}s iteration = 18 status = {"disabled":true,"errors":[],"sps":[]}
149602025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 18 reason = Timeout
149612025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
149622025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 18 reason = Timeout
149632025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
149642025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
149652025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
149662025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 18 reason = Timeout
149672025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
149682025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 18 reason = Timeout
149692025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
149702025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 18 reason = Timeout
149712025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
149722025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
149732025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
149742025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
149752025-12-16T03:07:39.315ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
149762025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
149772025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
149782025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
149792025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
149802025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
149812025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
149822025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
149832025-12-16T03:07:39.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
149842025-12-16T03:07:39.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
149852025-12-16T03:07:39.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
149862025-12-16T03:07:39.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
149872025-12-16T03:07:39.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
149882025-12-16T03:07:39.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
149892025-12-16T03:07:39.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
149902025-12-16T03:07:39.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
149912025-12-16T03:07:39.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
149922025-12-16T03:07:39.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
149932025-12-16T03:07:39.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
149942025-12-16T03:07:39.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
149952025-12-16T03:07:39.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
149962025-12-16T03:07:39.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
149972025-12-16T03:07:39.317ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149982025-12-16T03:07:39.318ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 143.877253ms iteration = 18 status = {}
149992025-12-16T03:07:39.318ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
150002025-12-16T03:07:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
150012025-12-16T03:07:39.319ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
150022025-12-16T03:07:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
150032025-12-16T03:07:39.320ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
150042025-12-16T03:07:39.320ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
150052025-12-16T03:07:39.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
150062025-12-16T03:07:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
150072025-12-16T03:07:39.320ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
150082025-12-16T03:07:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
150092025-12-16T03:07:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
150102025-12-16T03:07:39.324ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1466, tv_nsec: 934676472 } })
150112025-12-16T03:07:39.324ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
150122025-12-16T03:07:39.325ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1466, tv_nsec: 935769416 } })
150132025-12-16T03:07:39.329ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
150142025-12-16T03:07:39.329ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
150152025-12-16T03:07:39.329ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
150162025-12-16T03:07:39.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
150172025-12-16T03:07:39.362ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
150182025-12-16T03:07:39.362ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
150192025-12-16T03:07:39.362ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "81d2e555-8728-4ee0-bc73-56268e9a579b", "content-length": "26", "date": "Tue, 16 Dec 2025 03:07:39 GMT"} })
150202025-12-16T03:07:39.362ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
150212025-12-16T03:07:39.365ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
150222025-12-16T03:07:39.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150232025-12-16T03:07:39.374ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150242025-12-16T03:07:39.376ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150252025-12-16T03:07:39.377ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 61.789233ms iteration = 18 status = {"errors":[],"garbage_collect_requested":[]}
150262025-12-16T03:07:39.378ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
150272025-12-16T03:07:39.378ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
150282025-12-16T03:07:39.378ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
150292025-12-16T03:07:39.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
150302025-12-16T03:07:39.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
150312025-12-16T03:07:39.379ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
150322025-12-16T03:07:39.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
150332025-12-16T03:07:39.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
150342025-12-16T03:07:39.380ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150352025-12-16T03:07:39.381ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
150362025-12-16T03:07:39.381ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
150372025-12-16T03:07:39.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
150382025-12-16T03:07:39.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
150392025-12-16T03:07:39.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
150402025-12-16T03:07:39.383ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150412025-12-16T03:07:39.383ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150422025-12-16T03:07:39.384ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150432025-12-16T03:07:39.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150442025-12-16T03:07:39.385ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
150452025-12-16T03:07:39.385ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
150462025-12-16T03:07:39.385ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
150472025-12-16T03:07:39.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
150482025-12-16T03:07:39.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
150492025-12-16T03:07:39.385ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 70.24446ms iteration = 18 status = {"errors":[],"requests_created_ok":[]}
150502025-12-16T03:07:39.385ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
150512025-12-16T03:07:39.385ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
150522025-12-16T03:07:39.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
150532025-12-16T03:07:39.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
150542025-12-16T03:07:39.386ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
150552025-12-16T03:07:39.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
150562025-12-16T03:07:39.387ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 71.884461ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
150572025-12-16T03:07:39.388ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150582025-12-16T03:07:39.388ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
150592025-12-16T03:07:39.388ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
150602025-12-16T03:07:39.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
150612025-12-16T03:07:39.389ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150622025-12-16T03:07:39.389ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150632025-12-16T03:07:39.389ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
150642025-12-16T03:07:39.389ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
150652025-12-16T03:07:39.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
150662025-12-16T03:07:39.390ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150672025-12-16T03:07:39.390ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 75.457016ms iteration = 18 status = {"errors":[],"finish_invoked_ok":[]}
150682025-12-16T03:07:39.391ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
150692025-12-16T03:07:39.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
150702025-12-16T03:07:39.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
150712025-12-16T03:07:39.393ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
150722025-12-16T03:07:39.393ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
150732025-12-16T03:07:39.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
150742025-12-16T03:07:39.393ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
150752025-12-16T03:07:39.393ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
150762025-12-16T03:07:39.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
150772025-12-16T03:07:39.394ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
150782025-12-16T03:07:39.394ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
150792025-12-16T03:07:39.394ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
150802025-12-16T03:07:39.394ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
150812025-12-16T03:07:39.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
150822025-12-16T03:07:39.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
150832025-12-16T03:07:39.395ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
150842025-12-16T03:07:39.395ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
150852025-12-16T03:07:39.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
150862025-12-16T03:07:39.396ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 40cdf0c1-4d12-4355-9a49-17ccd3837b4c original_time_started = 2025-12-16 03:07:10.513292 UTC
150872025-12-16T03:07:39.396ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 81.101478ms iteration = 45 status = {"Loaded":{"collection_id":"40cdf0c1-4d12-4355-9a49-17ccd3837b4c","time_loaded":"2025-12-16T03:07:39.315142074Z","time_started":"2025-12-16T03:07:10.513292Z"}}
150882025-12-16T03:07:39.396ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
150892025-12-16T03:07:39.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
150902025-12-16T03:07:39.397ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150912025-12-16T03:07:39.397ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 82.343993ms iteration = 35 status = "NoSitrep"
150922025-12-16T03:07:39.397ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 82.631137ms iteration = 18 status = 0
150932025-12-16T03:07:39.397ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
150942025-12-16T03:07:39.397ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 82.662747ms iteration = 103 status = {"config_updated":false}
150952025-12-16T03:07:39.399ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
150962025-12-16T03:07:39.401ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
150972025-12-16T03:07:39.401ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
150982025-12-16T03:07:39.401ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 86.670708ms iteration = 18 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
150992025-12-16T03:07:39.402ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
151002025-12-16T03:07:39.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
151012025-12-16T03:07:39.403ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
151022025-12-16T03:07:39.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
151032025-12-16T03:07:39.405ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151042025-12-16T03:07:39.407ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
151052025-12-16T03:07:39.408ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
151062025-12-16T03:07:39.408ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 92.814996ms iteration = 35 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
151072025-12-16T03:07:39.411ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1467, tv_nsec: 21861287 } })
151082025-12-16T03:07:39.421ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
151092025-12-16T03:07:39.421ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 106.038493ms iteration = 52 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
151102025-12-16T03:07:39.434ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
151112025-12-16T03:07:39.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
151122025-12-16T03:07:39.437ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151132025-12-16T03:07:39.438ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 123.090938ms iteration = 52 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
151142025-12-16T03:07:39.440ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
151152025-12-16T03:07:39.440ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
151162025-12-16T03:07:39.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
151172025-12-16T03:07:39.446ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
151182025-12-16T03:07:39.446ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 131.09173ms iteration = 18 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
151192025-12-16T03:07:39.447ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
151202025-12-16T03:07:39.447ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1467, tv_nsec: 57783831 } })
151212025-12-16T03:07:39.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
151222025-12-16T03:07:39.450ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 135.454295ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
151232025-12-16T03:07:39.480ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
151242025-12-16T03:07:39.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
151252025-12-16T03:07:39.482ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
151262025-12-16T03:07:39.482ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 167.260189ms iteration = 27 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
151272025-12-16T03:07:39.483ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
151282025-12-16T03:07:39.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
151292025-12-16T03:07:39.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
151302025-12-16T03:07:39.489ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 174.177076ms iteration = 18 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
151312025-12-16T03:07:39.496ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
151322025-12-16T03:07:39.496ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
151332025-12-16T03:07:39.497ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
151342025-12-16T03:07:39.497ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
151352025-12-16T03:07:39.497ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
151362025-12-16T03:07:39.497ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
151372025-12-16T03:07:39.497ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
151382025-12-16T03:07:39.497ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
151392025-12-16T03:07:39.497ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
151402025-12-16T03:07:39.497ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
151412025-12-16T03:07:39.498ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
151422025-12-16T03:07:39.498ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
151432025-12-16T03:07:39.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
151442025-12-16T03:07:39.501ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
151452025-12-16T03:07:39.501ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
151462025-12-16T03:07:39.527ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
151472025-12-16T03:07:39.527ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
151482025-12-16T03:07:39.527ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "da6d332c-6677-4a33-b385-9000b4840c66", "content-length": "372", "date": "Tue, 16 Dec 2025 03:07:39 GMT"} })
151492025-12-16T03:07:39.527ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
151502025-12-16T03:07:39.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
151512025-12-16T03:07:39.542ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 226.707271ms iteration = 29 status = {"physical_disks_added":0}
151522025-12-16T03:07:39.544ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
151532025-12-16T03:07:39.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
151542025-12-16T03:07:39.546ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151552025-12-16T03:07:39.548ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
151562025-12-16T03:07:39.548ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
151572025-12-16T03:07:39.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
151582025-12-16T03:07:39.550ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151592025-12-16T03:07:39.552ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
151602025-12-16T03:07:39.552ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
151612025-12-16T03:07:39.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
151622025-12-16T03:07:39.578ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
151632025-12-16T03:07:39.578ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
151642025-12-16T03:07:39.579ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "01f562e9-e42a-4204-a7d1-7c54334c31ca", "date": "Tue, 16 Dec 2025 03:07:39 GMT"} })
151652025-12-16T03:07:39.579ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
151662025-12-16T03:07:39.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
151672025-12-16T03:07:39.581ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
151682025-12-16T03:07:39.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
151692025-12-16T03:07:39.584ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
151702025-12-16T03:07:39.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
151712025-12-16T03:07:39.586ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
151722025-12-16T03:07:39.588ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
151732025-12-16T03:07:39.588ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
151742025-12-16T03:07:39.588ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc4d0142-cdfa-4fd9-a8f8-0ad6ed27b15d", "content-length": "691", "date": "Tue, 16 Dec 2025 03:07:39 GMT"} })
151752025-12-16T03:07:39.588ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
151762025-12-16T03:07:39.589ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
151772025-12-16T03:07:39.589ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
151782025-12-16T03:07:39.589ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "e9a81006-edc9-4f93-a19c-dde4805b727b", "date": "Tue, 16 Dec 2025 03:07:39 GMT"} })
151792025-12-16T03:07:39.589ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 274.443137ms iteration = 18 status = {}
151802025-12-16T03:07:39.590ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 274.884482ms iteration = 27 status = {}
151812025-12-16T03:07:39.714ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "ec7d3a06-066f-411e-a518-0f9a57bf8081", "date": "Tue, 16 Dec 2025 03:07:39 GMT"} })
151822025-12-16T03:07:39.714ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
151832025-12-16T03:07:39.714ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
151842025-12-16T03:07:39.714ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "504c6e79-0b0f-46f8-8af4-fe9d3859e7f3", "content-length": "682", "date": "Tue, 16 Dec 2025 03:07:39 GMT"} })
151852025-12-16T03:07:39.714ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
151862025-12-16T03:07:39.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
151872025-12-16T03:07:39.718ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
151882025-12-16T03:07:39.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
151892025-12-16T03:07:39.721ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
151902025-12-16T03:07:39.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
151912025-12-16T03:07:39.723ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
151922025-12-16T03:07:39.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
151932025-12-16T03:07:39.731ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 415.944879ms iteration = 18 status = {}
151942025-12-16T03:07:40.190ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 3
151952025-12-16T03:07:40.190ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221
151962025-12-16T03:07:40.220ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
151972025-12-16T03:07:40.220ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:52190
151982025-12-16T03:07:40.220ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:52190 req_id = debd6eff-2fb5-409b-bcc3-bb2a472fb7dd uri = /metrics/producers version = 1.0.0
151992025-12-16T03:07:40.220ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:52190 req_id = debd6eff-2fb5-409b-bcc3-bb2a472fb7dd roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
152002025-12-16T03:07:40.221ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:52190 req_id = debd6eff-2fb5-409b-bcc3-bb2a472fb7dd resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
152012025-12-16T03:07:40.265ZINFOnexus: assigned collector to new producer collector_id = e5b81e4f-90b9-43ba-ba56-584fd121e601 file = nexus/src/app/oximeter.rs:76 producer_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
152022025-12-16T03:07:40.265ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 45065 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:52190 req_id = debd6eff-2fb5-409b-bcc3-bb2a472fb7dd response_code = 201 uri = /metrics/producers
152032025-12-16T03:07:40.265ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "debd6eff-2fb5-409b-bcc3-bb2a472fb7dd", "content-length": "41", "date": "Tue, 16 Dec 2025 03:07:40 GMT"} })
152042025-12-16T03:07:40.267ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
152052025-12-16T03:07:41.067ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 505 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 5604aa75-90d5-4089-8b6a-a5e9925efa10 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
152062025-12-16T03:07:44.317ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 104 reason = Timeout
152072025-12-16T03:07:44.317ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
152082025-12-16T03:07:44.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
152092025-12-16T03:07:44.318ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152102025-12-16T03:07:44.321ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
152112025-12-16T03:07:44.321ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
152122025-12-16T03:07:44.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
152132025-12-16T03:07:44.323ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
152142025-12-16T03:07:44.323ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.878397ms iteration = 104 status = {"config_updated":false}
152152025-12-16T03:07:49.317ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
152162025-12-16T03:07:49.317ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 53 reason = Timeout
152172025-12-16T03:07:49.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
152182025-12-16T03:07:49.317ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 105 reason = Timeout
152192025-12-16T03:07:49.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
152202025-12-16T03:07:49.317ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
152212025-12-16T03:07:49.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
152222025-12-16T03:07:49.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
152232025-12-16T03:07:49.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
152242025-12-16T03:07:49.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152252025-12-16T03:07:49.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
152262025-12-16T03:07:49.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
152272025-12-16T03:07:49.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152282025-12-16T03:07:49.321ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.250334ms iteration = 53 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
152292025-12-16T03:07:49.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
152302025-12-16T03:07:49.322ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
152312025-12-16T03:07:49.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
152322025-12-16T03:07:49.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
152332025-12-16T03:07:49.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
152342025-12-16T03:07:49.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
152352025-12-16T03:07:49.324ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
152362025-12-16T03:07:49.324ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.527206ms iteration = 105 status = {"config_updated":false}
152372025-12-16T03:07:49.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
152382025-12-16T03:07:49.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
152392025-12-16T03:07:49.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152402025-12-16T03:07:49.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
152412025-12-16T03:07:49.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
152422025-12-16T03:07:49.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
152432025-12-16T03:07:49.346ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
152442025-12-16T03:07:49.346ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.335222ms iteration = 53 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
152452025-12-16T03:07:51.066ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 321 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = befa49c3-c44f-43d8-806a-e40e32f832ca response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
152462025-12-16T03:07:54.316ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 46 reason = Timeout
152472025-12-16T03:07:54.316ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 36 reason = Timeout
152482025-12-16T03:07:54.316ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
152492025-12-16T03:07:54.316ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
152502025-12-16T03:07:54.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
152512025-12-16T03:07:54.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
152522025-12-16T03:07:54.316ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 36 reason = Timeout
152532025-12-16T03:07:54.316ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
152542025-12-16T03:07:54.316ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 106 reason = Timeout
152552025-12-16T03:07:54.316ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
152562025-12-16T03:07:54.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
152572025-12-16T03:07:54.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
152582025-12-16T03:07:54.318ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
152592025-12-16T03:07:54.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
152602025-12-16T03:07:54.318ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152612025-12-16T03:07:54.318ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152622025-12-16T03:07:54.318ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152632025-12-16T03:07:54.319ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152642025-12-16T03:07:54.322ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
152652025-12-16T03:07:54.322ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
152662025-12-16T03:07:54.322ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
152672025-12-16T03:07:54.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
152682025-12-16T03:07:54.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
152692025-12-16T03:07:54.323ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
152702025-12-16T03:07:54.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
152712025-12-16T03:07:54.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
152722025-12-16T03:07:54.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
152732025-12-16T03:07:54.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
152742025-12-16T03:07:54.324ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 8.210433ms iteration = 36 status = "NoSitrep"
152752025-12-16T03:07:54.325ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 40cdf0c1-4d12-4355-9a49-17ccd3837b4c original_time_started = 2025-12-16 03:07:10.513292 UTC
152762025-12-16T03:07:54.325ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
152772025-12-16T03:07:54.325ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 9.144406ms iteration = 46 status = {"Loaded":{"collection_id":"40cdf0c1-4d12-4355-9a49-17ccd3837b4c","time_loaded":"2025-12-16T03:07:54.315996503Z","time_started":"2025-12-16T03:07:10.513292Z"}}
152782025-12-16T03:07:54.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
152792025-12-16T03:07:54.325ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
152802025-12-16T03:07:54.325ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.609829ms iteration = 106 status = {"config_updated":false}
152812025-12-16T03:07:54.325ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
152822025-12-16T03:07:54.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
152832025-12-16T03:07:54.327ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152842025-12-16T03:07:54.328ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
152852025-12-16T03:07:54.329ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
152862025-12-16T03:07:54.329ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 12.295136ms iteration = 36 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
152872025-12-16T03:07:59.317ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 54 reason = Timeout
152882025-12-16T03:07:59.317ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
152892025-12-16T03:07:59.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
152902025-12-16T03:07:59.317ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 107 reason = Timeout
152912025-12-16T03:07:59.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
152922025-12-16T03:07:59.317ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
152932025-12-16T03:07:59.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
152942025-12-16T03:07:59.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
152952025-12-16T03:07:59.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
152962025-12-16T03:07:59.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152972025-12-16T03:07:59.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152982025-12-16T03:07:59.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
152992025-12-16T03:07:59.322ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
153002025-12-16T03:07:59.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
153012025-12-16T03:07:59.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
153022025-12-16T03:07:59.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
153032025-12-16T03:07:59.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
153042025-12-16T03:07:59.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
153052025-12-16T03:07:59.324ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
153062025-12-16T03:07:59.324ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.450014ms iteration = 107 status = {"config_updated":false}
153072025-12-16T03:07:59.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
153082025-12-16T03:07:59.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153092025-12-16T03:07:59.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
153102025-12-16T03:07:59.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
153112025-12-16T03:07:59.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
153122025-12-16T03:07:59.345ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
153132025-12-16T03:07:59.345ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.899124ms iteration = 54 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
153142025-12-16T03:07:59.355ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
153152025-12-16T03:07:59.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
153162025-12-16T03:07:59.357ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 39.790284ms iteration = 54 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
153172025-12-16T03:08:01.067ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 443 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = da29de07-102f-4edd-a132-21a20c245ed2 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
153182025-12-16T03:08:04.317ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 108 reason = Timeout
153192025-12-16T03:08:04.317ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
153202025-12-16T03:08:04.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
153212025-12-16T03:08:04.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153222025-12-16T03:08:04.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
153232025-12-16T03:08:04.322ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
153242025-12-16T03:08:04.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
153252025-12-16T03:08:04.324ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
153262025-12-16T03:08:04.324ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.833496ms iteration = 108 status = {"config_updated":false}
153272025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
153282025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
153292025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
153302025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
153312025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
153322025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
153332025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
153342025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
153352025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
153362025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
153372025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
153382025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
153392025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 10 reason = Timeout
153402025-12-16T03:08:09.175ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-12-16 02:58:09.175175505 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
153412025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
153422025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
153432025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
153442025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
153452025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 19 reason = Timeout
153462025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
153472025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
153482025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
153492025-12-16T03:08:09.175ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
153502025-12-16T03:08:09.175ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = dcc8c35d-7977-48b9-bbfa-35aa73a887e4 step component = ExternalNetworking step id = Ensure
153512025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
153522025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
153532025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
153542025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
153552025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
153562025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
153572025-12-16T03:08:09.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
153582025-12-16T03:08:09.176ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("49a51a59-9919-48ae-8182-4070a8430160.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6af60386-8771-4401-9d4d-e93774e519f0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 1496, tv_nsec: 785880917 } })
153592025-12-16T03:08:09.176ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1a45cd23-3d0a-43eb-adce-62c7b712baf1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1496, tv_nsec: 785899887 } })
153602025-12-16T03:08:09.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
153612025-12-16T03:08:09.176ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
153622025-12-16T03:08:09.176ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.54414ms iteration = 10 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
153632025-12-16T03:08:09.176ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
153642025-12-16T03:08:09.176ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
153652025-12-16T03:08:09.176ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.56655ms iteration = 10 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
153662025-12-16T03:08:09.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
153672025-12-16T03:08:09.178ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
153682025-12-16T03:08:09.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
153692025-12-16T03:08:09.178ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153702025-12-16T03:08:09.179ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-12-16 02:58:09.175175505 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
153712025-12-16T03:08:09.179ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1496, tv_nsec: 789176939 } })
153722025-12-16T03:08:09.179ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.095912ms iteration = 10 status = {"expiration":"2025-12-16T02:58:09.175175505Z","pruned":[]}
153732025-12-16T03:08:09.179ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153742025-12-16T03:08:09.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
153752025-12-16T03:08:09.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
153762025-12-16T03:08:09.180ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153772025-12-16T03:08:09.181ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1496, tv_nsec: 791159404 } })
153782025-12-16T03:08:09.181ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.374521ms iteration = 19 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
153792025-12-16T03:08:09.181ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
153802025-12-16T03:08:09.181ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1496, tv_nsec: 791884843 } })
153812025-12-16T03:08:09.183ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
153822025-12-16T03:08:09.183ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
153832025-12-16T03:08:09.183ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
153842025-12-16T03:08:09.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
153852025-12-16T03:08:09.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
153862025-12-16T03:08:09.186ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153872025-12-16T03:08:09.186ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
153882025-12-16T03:08:09.186ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
153892025-12-16T03:08:09.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
153902025-12-16T03:08:09.188ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
153912025-12-16T03:08:09.189ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153922025-12-16T03:08:09.193ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
153932025-12-16T03:08:09.195ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
153942025-12-16T03:08:09.195ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
153952025-12-16T03:08:09.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
153962025-12-16T03:08:09.196ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
153972025-12-16T03:08:09.199ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
153982025-12-16T03:08:09.201ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
153992025-12-16T03:08:09.204ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
154002025-12-16T03:08:09.206ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
154012025-12-16T03:08:09.209ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
154022025-12-16T03:08:09.212ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
154032025-12-16T03:08:09.215ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 23 reason = Timeout
154042025-12-16T03:08:09.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
154052025-12-16T03:08:09.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
154062025-12-16T03:08:09.217ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 5 reason = Timeout
154072025-12-16T03:08:09.217ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
154082025-12-16T03:08:09.217ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 10 reason = Timeout
154092025-12-16T03:08:09.217ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 10 reason = Timeout
154102025-12-16T03:08:09.217ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 10 reason = Timeout
154112025-12-16T03:08:09.217ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
154122025-12-16T03:08:09.217ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
154132025-12-16T03:08:09.217ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
154142025-12-16T03:08:09.217ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
154152025-12-16T03:08:09.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
154162025-12-16T03:08:09.218ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
154172025-12-16T03:08:09.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
154182025-12-16T03:08:09.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
154192025-12-16T03:08:09.218ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
154202025-12-16T03:08:09.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
154212025-12-16T03:08:09.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
154222025-12-16T03:08:09.218ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
154232025-12-16T03:08:09.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
154242025-12-16T03:08:09.219ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
154252025-12-16T03:08:09.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
154262025-12-16T03:08:09.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
154272025-12-16T03:08:09.224ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154282025-12-16T03:08:09.226ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154292025-12-16T03:08:09.228ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
154302025-12-16T03:08:09.228ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
154312025-12-16T03:08:09.228ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
154322025-12-16T03:08:09.228ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
154332025-12-16T03:08:09.228ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 10.776616ms iteration = 10 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
154342025-12-16T03:08:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
154352025-12-16T03:08:09.229ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
154362025-12-16T03:08:09.231ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
154372025-12-16T03:08:09.231ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
154382025-12-16T03:08:09.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
154392025-12-16T03:08:09.231ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154402025-12-16T03:08:09.234ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154412025-12-16T03:08:09.234ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 31 reason = Timeout
154422025-12-16T03:08:09.234ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
154432025-12-16T03:08:09.234ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 25.35\u{b5}s iteration = 31 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
154442025-12-16T03:08:09.234ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 10 reason = Timeout
154452025-12-16T03:08:09.234ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
154462025-12-16T03:08:09.234ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 10 reason = Timeout
154472025-12-16T03:08:09.234ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
154482025-12-16T03:08:09.234ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
154492025-12-16T03:08:09.234ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
154502025-12-16T03:08:09.234ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 10 reason = Timeout
154512025-12-16T03:08:09.234ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
154522025-12-16T03:08:09.234ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
154532025-12-16T03:08:09.234ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 10 reason = Timeout
154542025-12-16T03:08:09.234ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
154552025-12-16T03:08:09.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
154562025-12-16T03:08:09.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
154572025-12-16T03:08:09.235ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
154582025-12-16T03:08:09.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
154592025-12-16T03:08:09.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
154602025-12-16T03:08:09.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
154612025-12-16T03:08:09.238ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
154622025-12-16T03:08:09.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
154632025-12-16T03:08:09.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
154642025-12-16T03:08:09.239ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
154652025-12-16T03:08:09.239ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154662025-12-16T03:08:09.241ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 6.34546ms iteration = 10 status = {"errors":[],"found_port_ok":[]}
154672025-12-16T03:08:09.243ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
154682025-12-16T03:08:09.243ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
154692025-12-16T03:08:09.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
154702025-12-16T03:08:09.245ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
154712025-12-16T03:08:09.245ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 10.471623ms iteration = 10 status = {"by_rx":{},"error":null}
154722025-12-16T03:08:09.245ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
154732025-12-16T03:08:09.245ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154742025-12-16T03:08:09.247ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
154752025-12-16T03:08:09.248ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
154762025-12-16T03:08:09.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
154772025-12-16T03:08:09.248ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = dcc8c35d-7977-48b9-bbfa-35aa73a887e4 step component = SledList step id = Fetch
154782025-12-16T03:08:09.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
154792025-12-16T03:08:09.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
154802025-12-16T03:08:09.249ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
154812025-12-16T03:08:09.249ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
154822025-12-16T03:08:09.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
154832025-12-16T03:08:09.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154842025-12-16T03:08:09.252ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 17.405071ms iteration = 10 status = {"dispatched":[],"errors":[],"glob_version":"212.0.0","globs_reprocessed":{},"no_receivers":[]}
154852025-12-16T03:08:09.253ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154862025-12-16T03:08:09.256ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
154872025-12-16T03:08:09.256ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
154882025-12-16T03:08:09.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
154892025-12-16T03:08:09.257ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
154902025-12-16T03:08:09.257ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
154912025-12-16T03:08:09.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
154922025-12-16T03:08:09.259ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154932025-12-16T03:08:09.263ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
154942025-12-16T03:08:09.263ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
154952025-12-16T03:08:09.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
154962025-12-16T03:08:09.277ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
154972025-12-16T03:08:09.277ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
154982025-12-16T03:08:09.277ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
154992025-12-16T03:08:09.277ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
155002025-12-16T03:08:09.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
155012025-12-16T03:08:09.279ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
155022025-12-16T03:08:09.279ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
155032025-12-16T03:08:09.279ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
155042025-12-16T03:08:09.279ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
155052025-12-16T03:08:09.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
155062025-12-16T03:08:09.281ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155072025-12-16T03:08:09.282ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155082025-12-16T03:08:09.287ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
155092025-12-16T03:08:09.287ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
155102025-12-16T03:08:09.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
155112025-12-16T03:08:09.293ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
155122025-12-16T03:08:09.293ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
155132025-12-16T03:08:09.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
155142025-12-16T03:08:09.301ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
155152025-12-16T03:08:09.301ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
155162025-12-16T03:08:09.301ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3440a587-d3a4-47d0-965a-e10e3a5f45e0", "content-length": "26", "date": "Tue, 16 Dec 2025 03:08:09 GMT"} })
155172025-12-16T03:08:09.301ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
155182025-12-16T03:08:09.302ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
155192025-12-16T03:08:09.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
155202025-12-16T03:08:09.305ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
155212025-12-16T03:08:09.306ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1496, tv_nsec: 916432351 } })
155222025-12-16T03:08:09.312ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155232025-12-16T03:08:09.316ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
155242025-12-16T03:08:09.317ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
155252025-12-16T03:08:09.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
155262025-12-16T03:08:09.318ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
155272025-12-16T03:08:09.318ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
155282025-12-16T03:08:09.318ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 28 reason = Timeout
155292025-12-16T03:08:09.319ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
155302025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
155312025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 55 reason = Timeout
155322025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 19 reason = Timeout
155332025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 37 reason = Timeout
155342025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
155352025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
155362025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
155372025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
155382025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
155392025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 109 reason = Timeout
155402025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
155412025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 19 reason = Timeout
155422025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 19 reason = Timeout
155432025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
155442025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
155452025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
155462025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 19 reason = Timeout
155472025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
155482025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 19 reason = Timeout
155492025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
155502025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 19 reason = Timeout
155512025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
155522025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 28 reason = Timeout
155532025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
155542025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 30 reason = Timeout
155552025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
155562025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
155572025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 19 reason = Timeout
155582025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
155592025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
155602025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 47 reason = Timeout
155612025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 19 reason = Timeout
155622025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
155632025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 13.33\u{b5}s iteration = 19 status = {"disabled":true,"errors":[],"sps":[]}
155642025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 19 reason = Timeout
155652025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 37 reason = Timeout
155662025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
155672025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
155682025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
155692025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
155702025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
155712025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
155722025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
155732025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
155742025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
155752025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
155762025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
155772025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
155782025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
155792025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
155802025-12-16T03:08:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
155812025-12-16T03:08:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
155822025-12-16T03:08:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
155832025-12-16T03:08:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
155842025-12-16T03:08:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
155852025-12-16T03:08:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
155862025-12-16T03:08:09.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
155872025-12-16T03:08:09.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
155882025-12-16T03:08:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
155892025-12-16T03:08:09.320ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
155902025-12-16T03:08:09.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
155912025-12-16T03:08:09.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
155922025-12-16T03:08:09.322ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155932025-12-16T03:08:09.324ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155942025-12-16T03:08:09.326ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155952025-12-16T03:08:09.326ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155962025-12-16T03:08:09.326ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155972025-12-16T03:08:09.328ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155982025-12-16T03:08:09.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
155992025-12-16T03:08:09.329ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
156002025-12-16T03:08:09.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
156012025-12-16T03:08:09.330ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
156022025-12-16T03:08:09.330ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
156032025-12-16T03:08:09.330ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
156042025-12-16T03:08:09.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
156052025-12-16T03:08:09.330ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156062025-12-16T03:08:09.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
156072025-12-16T03:08:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
156082025-12-16T03:08:09.332ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
156092025-12-16T03:08:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
156102025-12-16T03:08:09.332ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
156112025-12-16T03:08:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
156122025-12-16T03:08:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
156132025-12-16T03:08:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
156142025-12-16T03:08:09.332ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
156152025-12-16T03:08:09.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
156162025-12-16T03:08:09.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
156172025-12-16T03:08:09.333ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
156182025-12-16T03:08:09.334ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156192025-12-16T03:08:09.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
156202025-12-16T03:08:09.335ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
156212025-12-16T03:08:09.335ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
156222025-12-16T03:08:09.336ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
156232025-12-16T03:08:09.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
156242025-12-16T03:08:09.337ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156252025-12-16T03:08:09.338ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
156262025-12-16T03:08:09.338ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
156272025-12-16T03:08:09.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
156282025-12-16T03:08:09.339ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 20.023974ms iteration = 19 status = {"errors":[],"requests_created_ok":[]}
156292025-12-16T03:08:09.341ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
156302025-12-16T03:08:09.341ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
156312025-12-16T03:08:09.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
156322025-12-16T03:08:09.341ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156332025-12-16T03:08:09.344ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
156342025-12-16T03:08:09.345ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
156352025-12-16T03:08:09.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
156362025-12-16T03:08:09.345ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 40cdf0c1-4d12-4355-9a49-17ccd3837b4c original_time_started = 2025-12-16 03:07:10.513292 UTC
156372025-12-16T03:08:09.345ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 26.142991ms iteration = 47 status = {"Loaded":{"collection_id":"40cdf0c1-4d12-4355-9a49-17ccd3837b4c","time_loaded":"2025-12-16T03:08:09.319353792Z","time_started":"2025-12-16T03:07:10.513292Z"}}
156382025-12-16T03:08:09.348ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156392025-12-16T03:08:09.348ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156402025-12-16T03:08:09.350ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
156412025-12-16T03:08:09.352ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
156422025-12-16T03:08:09.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
156432025-12-16T03:08:09.353ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 33.658046ms iteration = 37 status = "NoSitrep"
156442025-12-16T03:08:09.353ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
156452025-12-16T03:08:09.353ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
156462025-12-16T03:08:09.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
156472025-12-16T03:08:09.354ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
156482025-12-16T03:08:09.354ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
156492025-12-16T03:08:09.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
156502025-12-16T03:08:09.357ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
156512025-12-16T03:08:09.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
156522025-12-16T03:08:09.358ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
156532025-12-16T03:08:09.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
156542025-12-16T03:08:09.371ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1496, tv_nsec: 981739369 } })
156552025-12-16T03:08:09.372ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
156562025-12-16T03:08:09.372ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
156572025-12-16T03:08:09.372ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156582025-12-16T03:08:09.374ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1496, tv_nsec: 983633183 } })
156592025-12-16T03:08:09.374ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
156602025-12-16T03:08:09.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
156612025-12-16T03:08:09.375ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 199.959373ms iteration = 19 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
156622025-12-16T03:08:09.377ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
156632025-12-16T03:08:09.377ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
156642025-12-16T03:08:09.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
156652025-12-16T03:08:09.385ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
156662025-12-16T03:08:09.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
156672025-12-16T03:08:09.389ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 171.391942ms iteration = 5 status = {"nsuccess":5}
156682025-12-16T03:08:09.398ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 79.344235ms iteration = 19 status = 0
156692025-12-16T03:08:09.402ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156702025-12-16T03:08:09.404ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156712025-12-16T03:08:09.406ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
156722025-12-16T03:08:09.406ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
156732025-12-16T03:08:09.406ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156742025-12-16T03:08:09.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
156752025-12-16T03:08:09.409ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
156762025-12-16T03:08:09.409ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
156772025-12-16T03:08:09.409ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
156782025-12-16T03:08:09.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
156792025-12-16T03:08:09.410ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
156802025-12-16T03:08:09.412ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156812025-12-16T03:08:09.415ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
156822025-12-16T03:08:09.415ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
156832025-12-16T03:08:09.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
156842025-12-16T03:08:09.416ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
156852025-12-16T03:08:09.416ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
156862025-12-16T03:08:09.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
156872025-12-16T03:08:09.417ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
156882025-12-16T03:08:09.417ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 98.723991ms iteration = 37 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
156892025-12-16T03:08:09.418ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 99.334829ms iteration = 19 status = {"errors":[],"finish_invoked_ok":[]}
156902025-12-16T03:08:09.419ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156912025-12-16T03:08:09.419ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-12-16T02:59:05.020237Z, time_modified: 2025-12-16T02:59:05.020237Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }) }
156922025-12-16T03:08:09.419ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 244.466048ms iteration = 10 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]}},"default_endpoint":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
156932025-12-16T03:08:09.421ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156942025-12-16T03:08:09.424ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156952025-12-16T03:08:09.425ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156962025-12-16T03:08:09.425ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
156972025-12-16T03:08:09.425ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
156982025-12-16T03:08:09.425ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
156992025-12-16T03:08:09.425ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
157002025-12-16T03:08:09.425ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
157012025-12-16T03:08:09.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
157022025-12-16T03:08:09.425ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
157032025-12-16T03:08:09.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
157042025-12-16T03:08:09.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
157052025-12-16T03:08:09.426ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157062025-12-16T03:08:09.426ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
157072025-12-16T03:08:09.426ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
157082025-12-16T03:08:09.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
157092025-12-16T03:08:09.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
157102025-12-16T03:08:09.427ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
157112025-12-16T03:08:09.427ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
157122025-12-16T03:08:09.427ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
157132025-12-16T03:08:09.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
157142025-12-16T03:08:09.428ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157152025-12-16T03:08:09.429ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157162025-12-16T03:08:09.429ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
157172025-12-16T03:08:09.429ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
157182025-12-16T03:08:09.429ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-12-16 02:59:16.791641 UTC dns_group = internal generation = 1
157192025-12-16T03:08:09.429ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 254.512514ms iteration = 10 status = {"generation":1}
157202025-12-16T03:08:09.429ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 110.785334ms iteration = 55 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
157212025-12-16T03:08:09.432ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
157222025-12-16T03:08:09.432ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
157232025-12-16T03:08:09.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
157242025-12-16T03:08:09.433ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
157252025-12-16T03:08:09.433ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
157262025-12-16T03:08:09.433ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
157272025-12-16T03:08:09.433ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
157282025-12-16T03:08:09.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
157292025-12-16T03:08:09.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
157302025-12-16T03:08:09.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
157312025-12-16T03:08:09.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
157322025-12-16T03:08:09.435ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
157332025-12-16T03:08:09.436ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
157342025-12-16T03:08:09.436ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 117.057532ms iteration = 109 status = {"config_updated":false}
157352025-12-16T03:08:09.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157362025-12-16T03:08:09.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
157372025-12-16T03:08:09.436ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
157382025-12-16T03:08:09.436ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
157392025-12-16T03:08:09.437ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
157402025-12-16T03:08:09.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
157412025-12-16T03:08:09.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
157422025-12-16T03:08:09.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157432025-12-16T03:08:09.440ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
157442025-12-16T03:08:09.440ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
157452025-12-16T03:08:09.440ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-12-16 02:59:18.154440 UTC dns_group = external generation = 2
157462025-12-16T03:08:09.440ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 264.820834ms iteration = 10 status = {"generation":2}
157472025-12-16T03:08:09.440ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
157482025-12-16T03:08:09.440ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
157492025-12-16T03:08:09.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
157502025-12-16T03:08:09.443ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
157512025-12-16T03:08:09.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
157522025-12-16T03:08:09.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
157532025-12-16T03:08:09.447ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 127.789148ms iteration = 19 status = {"errors":[],"garbage_collect_requested":[]}
157542025-12-16T03:08:09.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
157552025-12-16T03:08:09.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
157562025-12-16T03:08:09.456ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
157572025-12-16T03:08:09.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
157582025-12-16T03:08:09.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
157592025-12-16T03:08:09.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
157602025-12-16T03:08:09.461ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
157612025-12-16T03:08:09.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
157622025-12-16T03:08:09.462ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = dcc8c35d-7977-48b9-bbfa-35aa73a887e4 step component = DeployNexusRecords step id = Ensure
157632025-12-16T03:08:09.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
157642025-12-16T03:08:09.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157652025-12-16T03:08:09.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
157662025-12-16T03:08:09.463ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
157672025-12-16T03:08:09.463ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 144.650382ms iteration = 19 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
157682025-12-16T03:08:09.466ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157692025-12-16T03:08:09.467ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157702025-12-16T03:08:09.467ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
157712025-12-16T03:08:09.467ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
157722025-12-16T03:08:09.467ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 249.590632ms iteration = 10 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
157732025-12-16T03:08:09.467ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
157742025-12-16T03:08:09.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
157752025-12-16T03:08:09.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
157762025-12-16T03:08:09.470ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
157772025-12-16T03:08:09.470ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
157782025-12-16T03:08:09.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
157792025-12-16T03:08:09.470ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
157802025-12-16T03:08:09.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
157812025-12-16T03:08:09.471ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
157822025-12-16T03:08:09.471ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
157832025-12-16T03:08:09.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
157842025-12-16T03:08:09.474ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
157852025-12-16T03:08:09.474ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157862025-12-16T03:08:09.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
157872025-12-16T03:08:09.476ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157882025-12-16T03:08:09.477ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 302.467031ms iteration = 19 status = {}
157892025-12-16T03:08:09.480ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
157902025-12-16T03:08:09.480ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
157912025-12-16T03:08:09.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
157922025-12-16T03:08:09.482ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
157932025-12-16T03:08:09.482ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 162.874743ms iteration = 55 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
157942025-12-16T03:08:09.482ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
157952025-12-16T03:08:09.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157962025-12-16T03:08:09.487ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
157972025-12-16T03:08:09.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
157982025-12-16T03:08:09.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
157992025-12-16T03:08:09.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158002025-12-16T03:08:09.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158012025-12-16T03:08:09.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158022025-12-16T03:08:09.497ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
158032025-12-16T03:08:09.497ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158042025-12-16T03:08:09.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158052025-12-16T03:08:09.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158062025-12-16T03:08:09.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158072025-12-16T03:08:09.504ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
158082025-12-16T03:08:09.504ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 185.282676ms iteration = 28 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
158092025-12-16T03:08:09.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158102025-12-16T03:08:09.508ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
158112025-12-16T03:08:09.508ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
158122025-12-16T03:08:09.508ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
158132025-12-16T03:08:09.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
158142025-12-16T03:08:09.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
158152025-12-16T03:08:09.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158162025-12-16T03:08:09.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158172025-12-16T03:08:09.513ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
158182025-12-16T03:08:09.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
158192025-12-16T03:08:09.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158202025-12-16T03:08:09.514ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
158212025-12-16T03:08:09.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158222025-12-16T03:08:09.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
158232025-12-16T03:08:09.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158242025-12-16T03:08:09.517ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 198.822179ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
158252025-12-16T03:08:09.520ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
158262025-12-16T03:08:09.520ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
158272025-12-16T03:08:09.521ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b79c8d2b-b57d-4d4a-bd1b-ea0a2db4077d", "content-length": "26", "date": "Tue, 16 Dec 2025 03:08:09 GMT"} })
158282025-12-16T03:08:09.521ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
158292025-12-16T03:08:09.522ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
158302025-12-16T03:08:09.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158312025-12-16T03:08:09.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158322025-12-16T03:08:09.523ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
158332025-12-16T03:08:09.538ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
158342025-12-16T03:08:09.548ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 228.17104ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
158352025-12-16T03:08:09.551ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = dcc8c35d-7977-48b9-bbfa-35aa73a887e4 step component = SledAgent step id = Ensure
158362025-12-16T03:08:09.555ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
158372025-12-16T03:08:09.555ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "ad01b83f-eb38-47df-a8b6-9009130ebe90", "date": "Tue, 16 Dec 2025 03:08:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
158382025-12-16T03:08:09.556ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
158392025-12-16T03:08:09.556ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158402025-12-16T03:08:09.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158412025-12-16T03:08:09.557ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "48d5ce75-ee1c-424f-983e-77fe6f32af8e", "date": "Tue, 16 Dec 2025 03:08:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
158422025-12-16T03:08:09.557ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
158432025-12-16T03:08:09.560ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
158442025-12-16T03:08:09.560ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 385.041898ms iteration = 12 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
158452025-12-16T03:08:09.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158462025-12-16T03:08:09.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158472025-12-16T03:08:09.563ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158482025-12-16T03:08:09.569ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
158492025-12-16T03:08:09.569ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158502025-12-16T03:08:09.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158512025-12-16T03:08:09.574ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158522025-12-16T03:08:09.578ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
158532025-12-16T03:08:09.578ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158542025-12-16T03:08:09.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158552025-12-16T03:08:09.580ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
158562025-12-16T03:08:09.580ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
158572025-12-16T03:08:09.580ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 260.944436ms iteration = 19 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
158582025-12-16T03:08:09.582ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158592025-12-16T03:08:09.583ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158602025-12-16T03:08:09.586ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158612025-12-16T03:08:09.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158622025-12-16T03:08:09.588ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158632025-12-16T03:08:09.593ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 274.481147ms iteration = 19 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
158642025-12-16T03:08:09.594ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
158652025-12-16T03:08:09.594ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158662025-12-16T03:08:09.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158672025-12-16T03:08:09.596ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158682025-12-16T03:08:09.600ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
158692025-12-16T03:08:09.600ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158702025-12-16T03:08:09.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158712025-12-16T03:08:09.604ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158722025-12-16T03:08:09.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158732025-12-16T03:08:09.605ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158742025-12-16T03:08:09.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158752025-12-16T03:08:09.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158762025-12-16T03:08:09.612ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
158772025-12-16T03:08:09.629ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 310.611505ms iteration = 30 status = {"physical_disks_added":0}
158782025-12-16T03:08:09.634ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
158792025-12-16T03:08:09.634ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
158802025-12-16T03:08:09.634ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "6553a6d8-0a3c-4ec7-b043-8687c7c7a35e", "content-length": "372", "date": "Tue, 16 Dec 2025 03:08:09 GMT"} })
158812025-12-16T03:08:09.634ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
158822025-12-16T03:08:09.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
158832025-12-16T03:08:09.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158842025-12-16T03:08:09.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158852025-12-16T03:08:09.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158862025-12-16T03:08:09.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158872025-12-16T03:08:09.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158882025-12-16T03:08:09.683ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
158892025-12-16T03:08:09.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158902025-12-16T03:08:09.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158912025-12-16T03:08:09.688ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158922025-12-16T03:08:09.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158932025-12-16T03:08:09.690ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158942025-12-16T03:08:09.696ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
158952025-12-16T03:08:09.696ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158962025-12-16T03:08:09.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158972025-12-16T03:08:09.699ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158982025-12-16T03:08:09.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158992025-12-16T03:08:09.701ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159002025-12-16T03:08:09.706ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
159012025-12-16T03:08:09.706ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159022025-12-16T03:08:09.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159032025-12-16T03:08:09.708ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159042025-12-16T03:08:09.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159052025-12-16T03:08:09.710ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159062025-12-16T03:08:09.714ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
159072025-12-16T03:08:09.714ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 496.596322ms iteration = 10 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
159082025-12-16T03:08:09.716ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
159092025-12-16T03:08:09.716ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159102025-12-16T03:08:09.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159112025-12-16T03:08:09.719ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159122025-12-16T03:08:09.719ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
159132025-12-16T03:08:09.719ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
159142025-12-16T03:08:09.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159152025-12-16T03:08:09.719ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "3d92fa72-cbf2-45ae-8087-c4bacb37c4b9", "date": "Tue, 16 Dec 2025 03:08:09 GMT"} })
159162025-12-16T03:08:09.719ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
159172025-12-16T03:08:09.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
159182025-12-16T03:08:09.720ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
159192025-12-16T03:08:09.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159202025-12-16T03:08:09.723ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "23bfb292-f3a7-4e7b-aa9b-c0fac211f2df", "date": "Tue, 16 Dec 2025 03:08:09 GMT"} }) sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
159212025-12-16T03:08:09.723ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
159222025-12-16T03:08:09.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
159232025-12-16T03:08:09.724ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
159242025-12-16T03:08:09.724ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159252025-12-16T03:08:09.724ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159262025-12-16T03:08:09.725ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = dcc8c35d-7977-48b9-bbfa-35aa73a887e4 step component = FirewallRules step id = Ensure
159272025-12-16T03:08:09.725ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159282025-12-16T03:08:09.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159292025-12-16T03:08:09.726ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
159302025-12-16T03:08:09.727ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
159312025-12-16T03:08:09.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
159322025-12-16T03:08:09.728ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159332025-12-16T03:08:09.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159342025-12-16T03:08:09.730ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159352025-12-16T03:08:09.730ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
159362025-12-16T03:08:09.730ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
159372025-12-16T03:08:09.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159382025-12-16T03:08:09.733ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "2eaee6b4-5ed9-46e3-85c1-5b5880035dbf", "date": "Tue, 16 Dec 2025 03:08:09 GMT"} })
159392025-12-16T03:08:09.733ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
159402025-12-16T03:08:09.733ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159412025-12-16T03:08:09.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159422025-12-16T03:08:09.734ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "38f98b98-50e5-4f7a-8863-510144fef412", "date": "Tue, 16 Dec 2025 03:08:09 GMT"} })
159432025-12-16T03:08:09.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159442025-12-16T03:08:09.735ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
159452025-12-16T03:08:09.735ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 500.723064ms iteration = 10 status = {"errors":[],"probes_by_sled":{"b1c76b61-acee-4be6-b582-973c938d6b6e":0}}
159462025-12-16T03:08:09.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159472025-12-16T03:08:09.735ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
159482025-12-16T03:08:09.736ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "b61a42f0-d840-45ab-b5c5-fc1e9a554c8a", "date": "Tue, 16 Dec 2025 03:08:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
159492025-12-16T03:08:09.736ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
159502025-12-16T03:08:09.737ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 418.481252ms iteration = 28 status = {}
159512025-12-16T03:08:09.737ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 29 reason = Dependency
159522025-12-16T03:08:09.737ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
159532025-12-16T03:08:09.737ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
159542025-12-16T03:08:09.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
159552025-12-16T03:08:09.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159562025-12-16T03:08:09.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159572025-12-16T03:08:09.740ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
159582025-12-16T03:08:09.740ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159592025-12-16T03:08:09.741ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae76113a-4663-4b6a-8dae-d6cc41578798", "content-length": "691", "date": "Tue, 16 Dec 2025 03:08:09 GMT"} })
159602025-12-16T03:08:09.741ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
159612025-12-16T03:08:09.741ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
159622025-12-16T03:08:09.741ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
159632025-12-16T03:08:09.742ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159642025-12-16T03:08:09.742ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "534b6b27-dd7b-42ab-b8e1-0f4e0930be06", "date": "Tue, 16 Dec 2025 03:08:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
159652025-12-16T03:08:09.742ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
159662025-12-16T03:08:09.743ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 424.412717ms iteration = 19 status = {}
159672025-12-16T03:08:09.744ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
159682025-12-16T03:08:09.744ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
159692025-12-16T03:08:09.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
159702025-12-16T03:08:09.748ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1497, tv_nsec: 358527842 } })
159712025-12-16T03:08:09.748ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159722025-12-16T03:08:09.749ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "cc9b35f9-d7b2-4c31-8356-ab6245c3db46", "date": "Tue, 16 Dec 2025 03:08:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
159732025-12-16T03:08:09.749ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
159742025-12-16T03:08:09.752ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
159752025-12-16T03:08:09.752ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 577.410735ms iteration = 12 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
159762025-12-16T03:08:09.752ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
159772025-12-16T03:08:09.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159782025-12-16T03:08:09.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159792025-12-16T03:08:09.753ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
159802025-12-16T03:08:09.753ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
159812025-12-16T03:08:09.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
159822025-12-16T03:08:09.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159832025-12-16T03:08:09.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159842025-12-16T03:08:09.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159852025-12-16T03:08:09.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159862025-12-16T03:08:09.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159872025-12-16T03:08:09.765ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
159882025-12-16T03:08:09.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159892025-12-16T03:08:09.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159902025-12-16T03:08:09.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159912025-12-16T03:08:09.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159922025-12-16T03:08:09.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159932025-12-16T03:08:09.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159942025-12-16T03:08:09.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159952025-12-16T03:08:09.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159962025-12-16T03:08:09.775ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159972025-12-16T03:08:09.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
159982025-12-16T03:08:09.777ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
159992025-12-16T03:08:09.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160002025-12-16T03:08:09.778ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160012025-12-16T03:08:09.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160022025-12-16T03:08:09.779ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
160032025-12-16T03:08:09.779ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
160042025-12-16T03:08:09.779ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 1b6756bc-cf88-4a11-b0e0-5181eb2cd30b parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
160052025-12-16T03:08:09.779ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160062025-12-16T03:08:09.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160072025-12-16T03:08:09.780ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160082025-12-16T03:08:09.781ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160092025-12-16T03:08:09.785ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
160102025-12-16T03:08:09.785ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160112025-12-16T03:08:09.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160122025-12-16T03:08:09.788ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
160132025-12-16T03:08:09.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160142025-12-16T03:08:09.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160152025-12-16T03:08:09.788ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
160162025-12-16T03:08:09.788ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
160172025-12-16T03:08:09.788ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 572.946068ms iteration = 23 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
160182025-12-16T03:08:09.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160192025-12-16T03:08:09.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160202025-12-16T03:08:09.791ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160212025-12-16T03:08:09.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160222025-12-16T03:08:09.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160232025-12-16T03:08:09.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160242025-12-16T03:08:09.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160252025-12-16T03:08:09.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160262025-12-16T03:08:09.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160272025-12-16T03:08:09.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160282025-12-16T03:08:09.805ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160292025-12-16T03:08:09.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160302025-12-16T03:08:09.807ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160312025-12-16T03:08:09.810ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1497, tv_nsec: 420533298 } })
160322025-12-16T03:08:09.813ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
160332025-12-16T03:08:09.813ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160342025-12-16T03:08:09.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160352025-12-16T03:08:09.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160362025-12-16T03:08:09.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160372025-12-16T03:08:09.819ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160382025-12-16T03:08:09.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160392025-12-16T03:08:09.822ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160402025-12-16T03:08:09.827ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
160412025-12-16T03:08:09.827ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160422025-12-16T03:08:09.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160432025-12-16T03:08:09.830ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
160442025-12-16T03:08:09.830ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
160452025-12-16T03:08:09.830ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "033e7105-3a0a-447a-8aff-970e80a082e5", "content-length": "372", "date": "Tue, 16 Dec 2025 03:08:09 GMT"} })
160462025-12-16T03:08:09.830ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
160472025-12-16T03:08:09.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
160482025-12-16T03:08:09.831ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160492025-12-16T03:08:09.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160502025-12-16T03:08:09.833ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160512025-12-16T03:08:09.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160522025-12-16T03:08:09.835ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160532025-12-16T03:08:09.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160542025-12-16T03:08:09.837ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160552025-12-16T03:08:09.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160562025-12-16T03:08:09.840ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160572025-12-16T03:08:09.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160582025-12-16T03:08:09.842ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160592025-12-16T03:08:09.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160602025-12-16T03:08:09.844ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160612025-12-16T03:08:09.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160622025-12-16T03:08:09.845ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160632025-12-16T03:08:09.855ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
160642025-12-16T03:08:09.855ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160652025-12-16T03:08:09.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160662025-12-16T03:08:09.855ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
160672025-12-16T03:08:09.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
160682025-12-16T03:08:09.857ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160692025-12-16T03:08:09.857ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
160702025-12-16T03:08:09.857ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
160712025-12-16T03:08:09.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160722025-12-16T03:08:09.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
160732025-12-16T03:08:09.859ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160742025-12-16T03:08:09.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160752025-12-16T03:08:09.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160762025-12-16T03:08:09.867ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
160772025-12-16T03:08:09.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160782025-12-16T03:08:09.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160792025-12-16T03:08:09.872ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
160802025-12-16T03:08:09.872ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
160812025-12-16T03:08:09.873ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
160822025-12-16T03:08:09.873ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
160832025-12-16T03:08:09.873ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
160842025-12-16T03:08:09.873ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
160852025-12-16T03:08:09.873ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
160862025-12-16T03:08:09.873ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
160872025-12-16T03:08:09.873ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
160882025-12-16T03:08:09.873ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
160892025-12-16T03:08:09.874ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
160902025-12-16T03:08:09.874ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
160912025-12-16T03:08:09.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
160922025-12-16T03:08:09.875ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }], Name("external-dns"): [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
160932025-12-16T03:08:09.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160942025-12-16T03:08:09.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160952025-12-16T03:08:09.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160962025-12-16T03:08:09.876ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
160972025-12-16T03:08:09.876ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
160982025-12-16T03:08:09.878ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
160992025-12-16T03:08:09.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161002025-12-16T03:08:09.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161012025-12-16T03:08:09.880ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
161022025-12-16T03:08:09.880ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
161032025-12-16T03:08:09.880ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
161042025-12-16T03:08:09.904ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
161052025-12-16T03:08:09.904ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
161062025-12-16T03:08:09.905ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9f72e88f-b966-410a-be9c-3ae56f3ccb2e", "date": "Tue, 16 Dec 2025 03:08:09 GMT"} })
161072025-12-16T03:08:09.905ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
161082025-12-16T03:08:09.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
161092025-12-16T03:08:09.907ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
161102025-12-16T03:08:09.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
161112025-12-16T03:08:09.911ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
161122025-12-16T03:08:09.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
161132025-12-16T03:08:09.914ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
161142025-12-16T03:08:09.914ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
161152025-12-16T03:08:09.919ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "650a225c-8d18-4bbb-95ed-d1a972efed0c", "date": "Tue, 16 Dec 2025 03:08:09 GMT"} })
161162025-12-16T03:08:09.920ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 183.458115ms iteration = 29 status = {}
161172025-12-16T03:08:09.949ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), time_created: 2025-12-16T02:59:15.195809Z, time_modified: 2025-12-16T02:59:15.195809Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
161182025-12-16T03:08:09.949ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
161192025-12-16T03:08:09.980ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161202025-12-16T03:08:09.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161212025-12-16T03:08:09.983ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161222025-12-16T03:08:09.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161232025-12-16T03:08:09.985ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161242025-12-16T03:08:09.989ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
161252025-12-16T03:08:09.989ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
161262025-12-16T03:08:09.993ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "1938ea41-8a7d-48fe-94e2-60580f86bfcf", "date": "Tue, 16 Dec 2025 03:08:09 GMT"} })
161272025-12-16T03:08:09.995ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
161282025-12-16T03:08:09.995ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = dcc8c35d-7977-48b9-bbfa-35aa73a887e4 step component = Dns step id = Ensure
161292025-12-16T03:08:09.995ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
161302025-12-16T03:08:09.995ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161312025-12-16T03:08:09.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161322025-12-16T03:08:09.996ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161332025-12-16T03:08:09.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161342025-12-16T03:08:09.998ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161352025-12-16T03:08:10.001ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
161362025-12-16T03:08:10.002ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
161372025-12-16T03:08:10.002ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
161382025-12-16T03:08:10.002ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
161392025-12-16T03:08:10.002ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161402025-12-16T03:08:10.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161412025-12-16T03:08:10.004ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161422025-12-16T03:08:10.007ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
161432025-12-16T03:08:10.007ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161442025-12-16T03:08:10.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161452025-12-16T03:08:10.009ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
161462025-12-16T03:08:10.009ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
161472025-12-16T03:08:10.009ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161482025-12-16T03:08:10.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161492025-12-16T03:08:10.011ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161502025-12-16T03:08:10.014ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
161512025-12-16T03:08:10.014ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161522025-12-16T03:08:10.014ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161532025-12-16T03:08:10.017ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
161542025-12-16T03:08:10.017ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
161552025-12-16T03:08:10.017ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
161562025-12-16T03:08:10.017ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161572025-12-16T03:08:10.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161582025-12-16T03:08:10.017ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161592025-12-16T03:08:10.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161602025-12-16T03:08:10.019ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161612025-12-16T03:08:10.022ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
161622025-12-16T03:08:10.023ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
161632025-12-16T03:08:10.023ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
161642025-12-16T03:08:10.023ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
161652025-12-16T03:08:10.023ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161662025-12-16T03:08:10.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161672025-12-16T03:08:10.024ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161682025-12-16T03:08:10.027ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
161692025-12-16T03:08:10.027ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161702025-12-16T03:08:10.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161712025-12-16T03:08:10.029ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
161722025-12-16T03:08:10.029ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
161732025-12-16T03:08:10.029ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161742025-12-16T03:08:10.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161752025-12-16T03:08:10.031ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161762025-12-16T03:08:10.034ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
161772025-12-16T03:08:10.034ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161782025-12-16T03:08:10.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161792025-12-16T03:08:10.035ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
161802025-12-16T03:08:10.035ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
161812025-12-16T03:08:10.035ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161822025-12-16T03:08:10.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161832025-12-16T03:08:10.036ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161842025-12-16T03:08:10.039ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
161852025-12-16T03:08:10.039ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161862025-12-16T03:08:10.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161872025-12-16T03:08:10.040ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161882025-12-16T03:08:10.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161892025-12-16T03:08:10.041ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161902025-12-16T03:08:10.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161912025-12-16T03:08:10.043ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161922025-12-16T03:08:10.046ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
161932025-12-16T03:08:10.048ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
161942025-12-16T03:08:10.048ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
161952025-12-16T03:08:10.048ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = dcc8c35d-7977-48b9-bbfa-35aa73a887e4 step component = OmicronZones step id = Cleanup
161962025-12-16T03:08:10.048ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = dcc8c35d-7977-48b9-bbfa-35aa73a887e4 step component = OmicronZones step id = Cleanup
161972025-12-16T03:08:10.048ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = dcc8c35d-7977-48b9-bbfa-35aa73a887e4 step component = PhysicalDisks step id = Cleanup
161982025-12-16T03:08:10.048ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = dcc8c35d-7977-48b9-bbfa-35aa73a887e4 step component = Clickhouse step id = Ensure
161992025-12-16T03:08:10.048ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = dcc8c35d-7977-48b9-bbfa-35aa73a887e4 step component = Clickhouse step id = Ensure
162002025-12-16T03:08:10.079ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
162012025-12-16T03:08:10.079ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 503, headers: {"content-type": "application/json", "x-request-id": "1192ba92-8b1b-4e1d-8666-7724453eb70d", "content-length": "94", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
162022025-12-16T03:08:10.080ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = dcc8c35d-7977-48b9-bbfa-35aa73a887e4 step component = SupportBundles step id = Cleanup
162032025-12-16T03:08:10.080ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
162042025-12-16T03:08:10.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
162052025-12-16T03:08:10.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162062025-12-16T03:08:10.084ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
162072025-12-16T03:08:10.084ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
162082025-12-16T03:08:10.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
162092025-12-16T03:08:10.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
162102025-12-16T03:08:10.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
162112025-12-16T03:08:10.087ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162122025-12-16T03:08:10.090ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
162132025-12-16T03:08:10.100ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "58168810-15cc-45c6-aba7-383cceda7ea0", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
162142025-12-16T03:08:10.100ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
162152025-12-16T03:08:10.100ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
162162025-12-16T03:08:10.101ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a64116f3-9116-4a5d-be82-228eaf5cce36", "content-length": "682", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
162172025-12-16T03:08:10.101ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
162182025-12-16T03:08:10.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
162192025-12-16T03:08:10.104ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
162202025-12-16T03:08:10.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
162212025-12-16T03:08:10.107ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
162222025-12-16T03:08:10.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
162232025-12-16T03:08:10.109ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
162242025-12-16T03:08:10.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
162252025-12-16T03:08:10.117ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 798.581317ms iteration = 19 status = {}
162262025-12-16T03:08:10.131ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = dcc8c35d-7977-48b9-bbfa-35aa73a887e4 step component = OmicronZones step id = Cleanup
162272025-12-16T03:08:10.131ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
162282025-12-16T03:08:10.131ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
162292025-12-16T03:08:10.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
162302025-12-16T03:08:10.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
162312025-12-16T03:08:10.132ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162322025-12-16T03:08:10.134ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
162332025-12-16T03:08:10.134ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
162342025-12-16T03:08:10.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
162352025-12-16T03:08:10.172ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
162362025-12-16T03:08:10.172ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(a91734db-9ac7-4793-9c32-ddbb21f8d986 (blueprint), false)
162372025-12-16T03:08:10.172ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = dcc8c35d-7977-48b9-bbfa-35aa73a887e4 step component = Cockroach step id = Ensure
162382025-12-16T03:08:10.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
162392025-12-16T03:08:10.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
162402025-12-16T03:08:10.182ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
162412025-12-16T03:08:10.182ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = dcc8c35d-7977-48b9-bbfa-35aa73a887e4 step component = MgsUpdates step id = Ensure
162422025-12-16T03:08:10.183ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.00793401s iteration = 11 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"2a1c436c-527d-485d-b63c-46274116c323","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"2a1c436c-527d-485d-b63c-46274116c323","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":4540,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":73159297,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":73159257,"secs":0}},"event_index":1,"execution_id":"2a1c436c-527d-485d-b63c-46274116c323","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":73183347,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":213466434,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":213466394,"secs":0}},"event_index":2,"execution_id":"2a1c436c-527d-485d-b63c-46274116c323","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":286665491,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":89553625,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":89553585,"secs":0}},"event_index":3,"execution_id":"2a1c436c-527d-485d-b63c-46274116c323","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":376237876,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":174291299,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":174291259,"secs":0}},"event_index":4,"execution_id":"2a1c436c-527d-485d-b63c-46274116c323","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":550552145,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":269688837,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":269688797,"secs":0}},"event_index":5,"execution_id":"2a1c436c-527d-485d-b63c-46274116c323","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":820254852,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":52547236,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":52547176,"secs":0}},"event_index":6,"execution_id":"2a1c436c-527d-485d-b63c-46274116c323","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":872816978,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3860,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":3830,"secs":0}},"event_index":7,"execution_id":"2a1c436c-527d-485d-b63c-46274116c323","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":872832788,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2250,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2210,"secs":0}},"event_index":8,"execution_id":"2a1c436c-527d-485d-b63c-46274116c323","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":872837708,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3330,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3301,"secs":0}},"event_index":9,"execution_id":"2a1c436c-527d-485d-b63c-46274116c323","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":872843179,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1589,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1560,"secs":0}},"event_index":10,"execution_id":"2a1c436c-527d-485d-b63c-46274116c323","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":872846599,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":32489492,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"warning","message":"failed to initialize single-node clickhouse database: Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"1192ba92-8b1b-4e1d-8666-7724453eb70d\\", \\"content-length\\": \\"94\\", \\"date\\": \\"Tue, 16 Dec 2025 03:08:10 GMT\\"}; value: Error { error_code: None, message: \\"Service Unavailable\\", request_id: \\"1192ba92-8b1b-4e1d-8666-7724453eb70d\\" }","metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":32489462,"secs":0}},"event_index":11,"execution_id":"2a1c436c-527d-485d-b63c-46274116c323","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":905342520,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":50471210,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":50471170,"secs":0}},"event_index":12,"execution_id":"2a1c436c-527d-485d-b63c-46274116c323","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":955824630,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":41526706,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":41526626,"secs":0}},"event_index":13,"execution_id":"2a1c436c-527d-485d-b63c-46274116c323","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":997362276,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":10092798,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":10092758,"secs":0}},"event_index":14,"execution_id":"2a1c436c-527d-485d-b63c-46274116c323","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":7468074,"secs":1}},{"data":{"attempt_elapsed":{"nanos":8831,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":8791,"secs":0}},"event_index":15,"execution_id":"2a1c436c-527d-485d-b63c-46274116c323","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":7487634,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986"}
162432025-12-16T03:08:10.183ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 29 reason = Dependency
162442025-12-16T03:08:10.185ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
162452025-12-16T03:08:10.185ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 11 reason = Dependency
162462025-12-16T03:08:10.185ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
162472025-12-16T03:08:10.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
162482025-12-16T03:08:10.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
162492025-12-16T03:08:10.187ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162502025-12-16T03:08:10.188ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162512025-12-16T03:08:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
162522025-12-16T03:08:10.190ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
162532025-12-16T03:08:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
162542025-12-16T03:08:10.190ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
162552025-12-16T03:08:10.190ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
162562025-12-16T03:08:10.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
162572025-12-16T03:08:10.194ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(e20dce38-fb3a-4fef-ab59-917046b281c6, 0), (d36b7584-5102-4a4f-888b-fa1c6fea6614, 0), (b97ed3f8-6719-4b04-8f47-be78e07de798, 0), (cce1609c-8305-491c-b4dd-4723203c6d3c, 0)] collection_id = e20dce38-fb3a-4fef-ab59-917046b281c6
162582025-12-16T03:08:10.194ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
162592025-12-16T03:08:10.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
162602025-12-16T03:08:10.234ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
162612025-12-16T03:08:10.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
162622025-12-16T03:08:10.237ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162632025-12-16T03:08:10.239ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
162642025-12-16T03:08:10.239ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
162652025-12-16T03:08:10.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
162662025-12-16T03:08:10.241ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
162672025-12-16T03:08:10.241ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 58.273338ms iteration = 29 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
162682025-12-16T03:08:10.344ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = e20dce38-fb3a-4fef-ab59-917046b281c6 file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
162692025-12-16T03:08:10.344ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
162702025-12-16T03:08:10.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
162712025-12-16T03:08:10.349ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(d36b7584-5102-4a4f-888b-fa1c6fea6614, 0), (b97ed3f8-6719-4b04-8f47-be78e07de798, 0), (cce1609c-8305-491c-b4dd-4723203c6d3c, 0), (377aeb05-9159-41c9-8113-9e905199b37b, 0)] collection_id = d36b7584-5102-4a4f-888b-fa1c6fea6614
162722025-12-16T03:08:10.349ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
162732025-12-16T03:08:10.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
162742025-12-16T03:08:10.487ZWARNnexus (datastore): Retryable transaction failure file = nexus/db-queries/src/transaction_retry.rs:165 retry_after (ms) = 40 transaction = inventory_delete_collection
162752025-12-16T03:08:10.560ZINFOnexus (datastore): transaction completed attempts = 2 file = nexus/db-queries/src/transaction_retry.rs:121 transaction = inventory_delete_collection
162762025-12-16T03:08:10.560ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = d36b7584-5102-4a4f-888b-fa1c6fea6614 file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
162772025-12-16T03:08:10.560ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
162782025-12-16T03:08:10.560ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
162792025-12-16T03:08:10.597ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(cce1609c-8305-491c-b4dd-4723203c6d3c, 0), (377aeb05-9159-41c9-8113-9e905199b37b, 0), (40cdf0c1-4d12-4355-9a49-17ccd3837b4c, 0)]
162802025-12-16T03:08:10.598ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1498, tv_nsec: 208121456 } })
162812025-12-16T03:08:10.630ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("20488683-5778-4090-b36e-e6f0d80a682b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("35dbaad1-8536-44a0-8a0a-35d7c39dcfbb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b57fa226-2428-4a54-b85d-60409e28b484.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("d812a3e2-48ae-4812-a1cc-7eb7623e4e00.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::5)) }], valid_until: Instant { tv_sec: 1498, tv_nsec: 240015080 } })
162822025-12-16T03:08:10.630ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
162832025-12-16T03:08:10.630ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
162842025-12-16T03:08:10.630ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
162852025-12-16T03:08:10.632ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ac88957-a7e2-4d58-837c-b22fc8f1ce8e", "content-length": "476", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
162862025-12-16T03:08:10.632ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
162872025-12-16T03:08:10.632ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0adba762-1473-43af-a630-f7cc74c3b401", "content-length": "734", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
162882025-12-16T03:08:10.632ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
162892025-12-16T03:08:10.633ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f1f03b7-a645-43ec-a15d-abd7b0cab7c2", "content-length": "111", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
162902025-12-16T03:08:10.633ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
162912025-12-16T03:08:10.633ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "23e1f11a-2a78-4f02-b417-43e0e91c97b9", "content-length": "111", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
162922025-12-16T03:08:10.633ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
162932025-12-16T03:08:10.633ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f463ac6-fc9d-48b8-b79e-22254d13103f", "content-length": "163", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
162942025-12-16T03:08:10.633ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
162952025-12-16T03:08:10.634ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "697017f0-38a0-4605-b1c5-7153139b7459", "content-length": "163", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
162962025-12-16T03:08:10.634ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
162972025-12-16T03:08:10.634ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f413d269-0cba-4f2b-bb1a-7089c3ee1759", "content-length": "166", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
162982025-12-16T03:08:10.634ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
162992025-12-16T03:08:10.635ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "46ddffc6-2e8a-4feb-bd9c-935fbf2aaa04", "content-length": "166", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163002025-12-16T03:08:10.635ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
163012025-12-16T03:08:10.635ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dfdd792e-90ad-4cfd-a9ec-ab3ba50a705e", "content-length": "702", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163022025-12-16T03:08:10.635ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
163032025-12-16T03:08:10.635ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8aabd4da-e56e-4b5b-8450-753e8b7b1bed", "content-length": "727", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163042025-12-16T03:08:10.635ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
163052025-12-16T03:08:10.636ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "759a8623-3b35-4de9-949a-5fc97c96149a", "content-length": "729", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163062025-12-16T03:08:10.636ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
163072025-12-16T03:08:10.636ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7506fa9-479f-418b-aad7-9047864c5aba", "content-length": "728", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163082025-12-16T03:08:10.636ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
163092025-12-16T03:08:10.636ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "44cfc79b-be30-4edd-a8be-cb0121fe4335", "content-length": "722", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163102025-12-16T03:08:10.636ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
163112025-12-16T03:08:10.636ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "72d8f9f3-2c92-4a47-80d7-3102c4705ea6", "content-length": "10", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163122025-12-16T03:08:10.636ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
163132025-12-16T03:08:10.637ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4a3f81c-c084-465a-9871-21ffe215d656", "content-length": "144", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163142025-12-16T03:08:10.637ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
163152025-12-16T03:08:10.637ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1538ddc-49ae-4f07-b10f-15a957c1f613", "content-length": "147", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163162025-12-16T03:08:10.637ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
163172025-12-16T03:08:10.637ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f85d75a-1741-438f-805e-f728302cfdaa", "content-length": "109", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163182025-12-16T03:08:10.637ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
163192025-12-16T03:08:10.638ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d8b4e861-dcb1-4395-af02-77ae4bf300c4", "content-length": "109", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163202025-12-16T03:08:10.638ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
163212025-12-16T03:08:10.639ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "994d4316-3155-4aa7-baf4-cd6f4d4ab55d", "content-length": "163", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163222025-12-16T03:08:10.639ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
163232025-12-16T03:08:10.639ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0216527-64a6-427c-9fe8-c9da9a916ca4", "content-length": "163", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163242025-12-16T03:08:10.639ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
163252025-12-16T03:08:10.640ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "835d4360-20f0-4e96-b1dc-debd44e2f2ea", "content-length": "166", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163262025-12-16T03:08:10.640ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
163272025-12-16T03:08:10.640ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "33fd29dd-299f-4ed2-99dc-0c784dcd6b9c", "content-length": "166", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163282025-12-16T03:08:10.640ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
163292025-12-16T03:08:10.640ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a2b7a2a-2c19-4ff5-bd01-1ead69f530d7", "content-length": "702", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163302025-12-16T03:08:10.640ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
163312025-12-16T03:08:10.641ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "351853ba-426e-4501-b36e-80dfb6f0e693", "content-length": "727", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163322025-12-16T03:08:10.641ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
163332025-12-16T03:08:10.641ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7025cd55-52fd-4485-b341-b9b380a75408", "content-length": "729", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163342025-12-16T03:08:10.641ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
163352025-12-16T03:08:10.641ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d5d204d-a934-48f6-9aa9-93a46b67d74b", "content-length": "728", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163362025-12-16T03:08:10.641ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163372025-12-16T03:08:10.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
163382025-12-16T03:08:10.643ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163392025-12-16T03:08:10.646ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
163402025-12-16T03:08:10.646ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163412025-12-16T03:08:10.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
163422025-12-16T03:08:10.647ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163432025-12-16T03:08:10.649ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
163442025-12-16T03:08:10.649ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163452025-12-16T03:08:10.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
163462025-12-16T03:08:10.682ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
163472025-12-16T03:08:10.682ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
163482025-12-16T03:08:10.823ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "488b5ae1-a9b2-463c-97e9-6d6ec5e95de5", "content-length": "95612", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163492025-12-16T03:08:10.826ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
163502025-12-16T03:08:10.826ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
163512025-12-16T03:08:10.826ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
163522025-12-16T03:08:10.826ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
163532025-12-16T03:08:10.826ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
163542025-12-16T03:08:10.826ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
163552025-12-16T03:08:10.826ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
163562025-12-16T03:08:10.826ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
163572025-12-16T03:08:10.826ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e209b737-b67c-4eac-94fa-3854b5a2511b", "content-length": "64", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163582025-12-16T03:08:10.826ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2eaba6b-d8d9-46c2-a583-6a25010311fb", "content-length": "64", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163592025-12-16T03:08:10.826ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
163602025-12-16T03:08:10.826ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
163612025-12-16T03:08:10.826ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1f357e2-98a8-47aa-822e-850c0dae2b94", "content-length": "64", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163622025-12-16T03:08:10.826ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
163632025-12-16T03:08:10.826ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3572dc7-37ed-4191-9809-9e1ed40daa00", "content-length": "64", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163642025-12-16T03:08:10.826ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3383582-4fb4-4549-9173-5d718395752a", "content-length": "64", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163652025-12-16T03:08:10.826ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
163662025-12-16T03:08:10.826ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
163672025-12-16T03:08:10.871ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba064276-20a9-45c8-8396-cb272940bfab", "content-length": "435856", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163682025-12-16T03:08:10.871ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "145adf55-bd16-4756-aae0-dcdd0b1cb337", "content-length": "405248", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163692025-12-16T03:08:10.872ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "98e0af7d-60ae-4a6a-ac98-d31d90f4c24f", "content-length": "403554", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163702025-12-16T03:08:10.872ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "0babb27b-2a40-48fe-a06b-022054ddf4b3", "content-length": "407090", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163712025-12-16T03:08:10.872ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b667f33-595c-4019-bb47-cd4a001ca67c", "content-length": "406008", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163722025-12-16T03:08:10.876ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
163732025-12-16T03:08:10.876ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
163742025-12-16T03:08:10.876ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
163752025-12-16T03:08:10.876ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
163762025-12-16T03:08:10.876ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
163772025-12-16T03:08:10.908ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c (omicron_zone)
163782025-12-16T03:08:10.908ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
163792025-12-16T03:08:10.912ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "8556afb1-b339-47be-9634-c8ab89202fac", "content-length": "125", "date": "Tue, 16 Dec 2025 03:08:10 GMT"} })
163802025-12-16T03:08:10.914ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
163812025-12-16T03:08:11.012ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
163822025-12-16T03:08:11.013ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
163832025-12-16T03:08:11.013ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "585b1887-954b-4706-aa18-bdee100db73d", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:08:11 GMT"} })
163842025-12-16T03:08:11.013ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
163852025-12-16T03:08:11.016ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
163862025-12-16T03:08:11.016ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
163872025-12-16T03:08:11.016ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b051329-d6b0-4f8f-ab73-6ac42985771d", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:08:11 GMT"} })
163882025-12-16T03:08:11.016ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
163892025-12-16T03:08:11.018ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
163902025-12-16T03:08:11.018ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
163912025-12-16T03:08:11.018ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "699ffe33-d807-4146-b4ad-4a1ba434b047", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:08:11 GMT"} })
163922025-12-16T03:08:11.018ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
163932025-12-16T03:08:11.021ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
163942025-12-16T03:08:11.021ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
163952025-12-16T03:08:11.023ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163962025-12-16T03:08:11.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
163972025-12-16T03:08:11.025ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163982025-12-16T03:08:11.028ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
163992025-12-16T03:08:11.028ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
164002025-12-16T03:08:11.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
164012025-12-16T03:08:11.068ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 405 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 09543a4a-6d6b-41f8-b999-8601538e3ea5 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
164022025-12-16T03:08:11.245ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = a8a48239-3eed-426a-8ff8-7f8d17a22586 file = nexus/db-queries/src/db/datastore/inventory.rs:1613
164032025-12-16T03:08:11.245ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = a8a48239-3eed-426a-8ff8-7f8d17a22586 time_started = 2025-12-16 03:08:10.630880 UTC
164042025-12-16T03:08:11.245ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 32 reason = Dependency
164052025-12-16T03:08:11.245ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
164062025-12-16T03:08:11.245ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 48 reason = Dependency
164072025-12-16T03:08:11.245ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 32.241\u{b5}s iteration = 32 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
164082025-12-16T03:08:11.245ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
164092025-12-16T03:08:11.245ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.060381325s iteration = 11 status = {"collection_id":"a8a48239-3eed-426a-8ff8-7f8d17a22586","time_done":"2025-12-16 03:08:11.021240 UTC","time_started":"2025-12-16 03:08:10.630880 UTC"}
164102025-12-16T03:08:11.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
164112025-12-16T03:08:11.250ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164122025-12-16T03:08:11.255ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
164132025-12-16T03:08:11.255ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
164142025-12-16T03:08:11.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
164152025-12-16T03:08:11.258ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
164162025-12-16T03:08:11.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
164172025-12-16T03:08:11.323ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 14 reason = Dependency
164182025-12-16T03:08:11.323ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 33 reason = Dependency
164192025-12-16T03:08:11.323ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 78.304792ms iteration = 48 status = {"Loaded":{"collection_id":"1845c551-ac2c-40d7-ad4c-fd64ea891f40","time_loaded":"2025-12-16T03:08:11.245514786Z","time_started":"2025-12-16T03:08:10.631572Z"}}
164202025-12-16T03:08:11.323ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 24 reason = Dependency
164212025-12-16T03:08:11.323ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
164222025-12-16T03:08:11.323ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 31 reason = Dependency
164232025-12-16T03:08:11.323ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 29.05\u{b5}s iteration = 33 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
164242025-12-16T03:08:11.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164252025-12-16T03:08:11.323ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
164262025-12-16T03:08:11.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
164272025-12-16T03:08:11.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
164282025-12-16T03:08:11.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164292025-12-16T03:08:11.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
164302025-12-16T03:08:11.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164312025-12-16T03:08:11.326ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164322025-12-16T03:08:11.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164332025-12-16T03:08:11.329ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
164342025-12-16T03:08:11.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
164352025-12-16T03:08:11.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
164362025-12-16T03:08:11.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
164372025-12-16T03:08:11.330ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
164382025-12-16T03:08:11.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
164392025-12-16T03:08:11.330ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
164402025-12-16T03:08:11.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164412025-12-16T03:08:11.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164422025-12-16T03:08:11.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164432025-12-16T03:08:11.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164442025-12-16T03:08:11.333ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
164452025-12-16T03:08:11.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
164462025-12-16T03:08:11.334ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
164472025-12-16T03:08:11.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
164482025-12-16T03:08:11.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
164492025-12-16T03:08:11.335ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164502025-12-16T03:08:11.336ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
164512025-12-16T03:08:11.336ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164522025-12-16T03:08:11.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164532025-12-16T03:08:11.337ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
164542025-12-16T03:08:11.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
164552025-12-16T03:08:11.337ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
164562025-12-16T03:08:11.337ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
164572025-12-16T03:08:11.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
164582025-12-16T03:08:11.339ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164592025-12-16T03:08:11.341ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
164602025-12-16T03:08:11.341ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
164612025-12-16T03:08:11.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
164622025-12-16T03:08:11.343ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164632025-12-16T03:08:11.347ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
164642025-12-16T03:08:11.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
164652025-12-16T03:08:11.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
164662025-12-16T03:08:11.353ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
164672025-12-16T03:08:11.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
164682025-12-16T03:08:11.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
164692025-12-16T03:08:11.356ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164702025-12-16T03:08:11.358ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
164712025-12-16T03:08:11.358ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
164722025-12-16T03:08:11.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
164732025-12-16T03:08:11.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164742025-12-16T03:08:11.363ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
164752025-12-16T03:08:11.363ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
164762025-12-16T03:08:11.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
164772025-12-16T03:08:11.366ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
164782025-12-16T03:08:11.366ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 42.312316ms iteration = 14 status = {"blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","inventory_collection_id":"1845c551-ac2c-40d7-ad4c-fd64ea891f40","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
164792025-12-16T03:08:11.402ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 78.16482ms iteration = 31 status = {"physical_disks_added":0}
164802025-12-16T03:08:11.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164812025-12-16T03:08:11.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164822025-12-16T03:08:11.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164832025-12-16T03:08:11.462ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
164842025-12-16T03:08:11.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164852025-12-16T03:08:11.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164862025-12-16T03:08:11.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164872025-12-16T03:08:11.467ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
164882025-12-16T03:08:11.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164892025-12-16T03:08:11.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164902025-12-16T03:08:11.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164912025-12-16T03:08:11.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164922025-12-16T03:08:11.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164932025-12-16T03:08:11.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164942025-12-16T03:08:11.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164952025-12-16T03:08:11.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
164962025-12-16T03:08:11.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164972025-12-16T03:08:11.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
164982025-12-16T03:08:11.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
164992025-12-16T03:08:11.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165002025-12-16T03:08:11.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165012025-12-16T03:08:11.549ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
165022025-12-16T03:08:11.549ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165032025-12-16T03:08:11.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165042025-12-16T03:08:11.551ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165052025-12-16T03:08:11.555ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
165062025-12-16T03:08:11.555ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165072025-12-16T03:08:11.555ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165082025-12-16T03:08:11.558ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165092025-12-16T03:08:11.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165102025-12-16T03:08:11.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165112025-12-16T03:08:11.564ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
165122025-12-16T03:08:11.564ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165132025-12-16T03:08:11.564ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165142025-12-16T03:08:11.567ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165152025-12-16T03:08:11.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165162025-12-16T03:08:11.569ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165172025-12-16T03:08:11.573ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
165182025-12-16T03:08:11.573ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165192025-12-16T03:08:11.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165202025-12-16T03:08:11.576ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165212025-12-16T03:08:11.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165222025-12-16T03:08:11.578ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165232025-12-16T03:08:11.582ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
165242025-12-16T03:08:11.582ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165252025-12-16T03:08:11.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165262025-12-16T03:08:11.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165272025-12-16T03:08:11.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165282025-12-16T03:08:11.609ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165292025-12-16T03:08:11.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165302025-12-16T03:08:11.611ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165312025-12-16T03:08:11.614ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
165322025-12-16T03:08:11.614ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165332025-12-16T03:08:11.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165342025-12-16T03:08:11.615ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165352025-12-16T03:08:11.618ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
165362025-12-16T03:08:11.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165372025-12-16T03:08:11.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165382025-12-16T03:08:11.621ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165392025-12-16T03:08:11.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165402025-12-16T03:08:11.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165412025-12-16T03:08:11.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165422025-12-16T03:08:11.625ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165432025-12-16T03:08:11.628ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
165442025-12-16T03:08:11.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165452025-12-16T03:08:11.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165462025-12-16T03:08:11.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165472025-12-16T03:08:11.632ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
165482025-12-16T03:08:11.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165492025-12-16T03:08:11.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165502025-12-16T03:08:11.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165512025-12-16T03:08:11.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165522025-12-16T03:08:11.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165532025-12-16T03:08:11.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165542025-12-16T03:08:11.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165552025-12-16T03:08:11.640ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
165562025-12-16T03:08:11.641ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165572025-12-16T03:08:11.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165582025-12-16T03:08:11.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165592025-12-16T03:08:11.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165602025-12-16T03:08:11.647ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165612025-12-16T03:08:11.652ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
165622025-12-16T03:08:11.653ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165632025-12-16T03:08:11.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165642025-12-16T03:08:11.655ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165652025-12-16T03:08:11.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165662025-12-16T03:08:11.656ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165672025-12-16T03:08:11.659ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
165682025-12-16T03:08:11.659ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165692025-12-16T03:08:11.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165702025-12-16T03:08:11.661ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165712025-12-16T03:08:11.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165722025-12-16T03:08:11.663ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165732025-12-16T03:08:11.666ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
165742025-12-16T03:08:11.666ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165752025-12-16T03:08:11.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165762025-12-16T03:08:11.668ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165772025-12-16T03:08:11.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165782025-12-16T03:08:11.669ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165792025-12-16T03:08:11.672ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
165802025-12-16T03:08:11.672ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165812025-12-16T03:08:11.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165822025-12-16T03:08:11.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165832025-12-16T03:08:11.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165842025-12-16T03:08:11.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165852025-12-16T03:08:11.678ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
165862025-12-16T03:08:11.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165872025-12-16T03:08:11.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165882025-12-16T03:08:11.713ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
165892025-12-16T03:08:11.713ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
165902025-12-16T03:08:11.713ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 0db4388a-768f-4048-af3b-12327d43f89a parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
165912025-12-16T03:08:11.713ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165922025-12-16T03:08:11.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165932025-12-16T03:08:11.715ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165942025-12-16T03:08:11.718ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
165952025-12-16T03:08:11.718ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165962025-12-16T03:08:11.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165972025-12-16T03:08:11.721ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
165982025-12-16T03:08:11.721ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
165992025-12-16T03:08:11.721ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 397.84289ms iteration = 24 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
166002025-12-16T03:08:14.317ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 110 reason = Timeout
166012025-12-16T03:08:14.317ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
166022025-12-16T03:08:14.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
166032025-12-16T03:08:14.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166042025-12-16T03:08:14.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
166052025-12-16T03:08:14.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
166062025-12-16T03:08:14.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
166072025-12-16T03:08:14.324ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
166082025-12-16T03:08:14.324ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.267102ms iteration = 110 status = {"config_updated":false}
166092025-12-16T03:08:19.317ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 56 reason = Timeout
166102025-12-16T03:08:19.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
166112025-12-16T03:08:19.317ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
166122025-12-16T03:08:19.317ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 111 reason = Timeout
166132025-12-16T03:08:19.317ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
166142025-12-16T03:08:19.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
166152025-12-16T03:08:19.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
166162025-12-16T03:08:19.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
166172025-12-16T03:08:19.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
166182025-12-16T03:08:19.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166192025-12-16T03:08:19.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166202025-12-16T03:08:19.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
166212025-12-16T03:08:19.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
166222025-12-16T03:08:19.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
166232025-12-16T03:08:19.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
166242025-12-16T03:08:19.322ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
166252025-12-16T03:08:19.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
166262025-12-16T03:08:19.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
166272025-12-16T03:08:19.324ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
166282025-12-16T03:08:19.324ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.717008ms iteration = 111 status = {"config_updated":false}
166292025-12-16T03:08:19.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
166302025-12-16T03:08:19.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166312025-12-16T03:08:19.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
166322025-12-16T03:08:19.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
166332025-12-16T03:08:19.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
166342025-12-16T03:08:19.344ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
166352025-12-16T03:08:19.344ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.759272ms iteration = 56 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
166362025-12-16T03:08:19.354ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
166372025-12-16T03:08:19.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
166382025-12-16T03:08:19.355ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 38.802492ms iteration = 56 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
166392025-12-16T03:08:21.067ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 278 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 4244118d-b837-495f-bda3-408e9bf5888c response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
166402025-12-16T03:08:24.316ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 49 reason = Timeout
166412025-12-16T03:08:24.316ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 38 reason = Timeout
166422025-12-16T03:08:24.316ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
166432025-12-16T03:08:24.316ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 38 reason = Timeout
166442025-12-16T03:08:24.316ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
166452025-12-16T03:08:24.316ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
166462025-12-16T03:08:24.317ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 112 reason = Timeout
166472025-12-16T03:08:24.317ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
166482025-12-16T03:08:24.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
166492025-12-16T03:08:24.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
166502025-12-16T03:08:24.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
166512025-12-16T03:08:24.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
166522025-12-16T03:08:24.318ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
166532025-12-16T03:08:24.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
166542025-12-16T03:08:24.319ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166552025-12-16T03:08:24.319ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166562025-12-16T03:08:24.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166572025-12-16T03:08:24.320ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166582025-12-16T03:08:24.322ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
166592025-12-16T03:08:24.322ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
166602025-12-16T03:08:24.322ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
166612025-12-16T03:08:24.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
166622025-12-16T03:08:24.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
166632025-12-16T03:08:24.323ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
166642025-12-16T03:08:24.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
166652025-12-16T03:08:24.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
166662025-12-16T03:08:24.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
166672025-12-16T03:08:24.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
166682025-12-16T03:08:24.325ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 8.282205ms iteration = 38 status = "NoSitrep"
166692025-12-16T03:08:24.325ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 1845c551-ac2c-40d7-ad4c-fd64ea891f40 original_time_started = 2025-12-16 03:08:10.631572 UTC
166702025-12-16T03:08:24.325ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.536818ms iteration = 49 status = {"Loaded":{"collection_id":"1845c551-ac2c-40d7-ad4c-fd64ea891f40","time_loaded":"2025-12-16T03:08:24.316855217Z","time_started":"2025-12-16T03:08:10.631572Z"}}
166712025-12-16T03:08:24.325ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
166722025-12-16T03:08:24.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
166732025-12-16T03:08:24.326ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
166742025-12-16T03:08:24.326ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 9.107176ms iteration = 112 status = {"config_updated":false}
166752025-12-16T03:08:24.326ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
166762025-12-16T03:08:24.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
166772025-12-16T03:08:24.327ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166782025-12-16T03:08:24.329ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
166792025-12-16T03:08:24.329ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
166802025-12-16T03:08:24.329ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 12.886473ms iteration = 38 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
166812025-12-16T03:08:29.317ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
166822025-12-16T03:08:29.317ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 57 reason = Timeout
166832025-12-16T03:08:29.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
166842025-12-16T03:08:29.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
166852025-12-16T03:08:29.317ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 113 reason = Timeout
166862025-12-16T03:08:29.317ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
166872025-12-16T03:08:29.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
166882025-12-16T03:08:29.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
166892025-12-16T03:08:29.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
166902025-12-16T03:08:29.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166912025-12-16T03:08:29.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
166922025-12-16T03:08:29.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166932025-12-16T03:08:29.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
166942025-12-16T03:08:29.320ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.897719ms iteration = 57 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
166952025-12-16T03:08:29.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
166962025-12-16T03:08:29.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
166972025-12-16T03:08:29.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
166982025-12-16T03:08:29.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
166992025-12-16T03:08:29.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
167002025-12-16T03:08:29.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
167012025-12-16T03:08:29.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
167022025-12-16T03:08:29.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
167032025-12-16T03:08:29.325ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
167042025-12-16T03:08:29.325ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.249905ms iteration = 113 status = {"config_updated":false}
167052025-12-16T03:08:29.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167062025-12-16T03:08:29.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
167072025-12-16T03:08:29.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
167082025-12-16T03:08:29.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
167092025-12-16T03:08:29.345ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
167102025-12-16T03:08:29.345ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.351199ms iteration = 57 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
167112025-12-16T03:08:31.068ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 318 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 3ee8ad2d-b180-404f-b68d-de887dff8486 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
167122025-12-16T03:08:34.317ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 114 reason = Timeout
167132025-12-16T03:08:34.317ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
167142025-12-16T03:08:34.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
167152025-12-16T03:08:34.320ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167162025-12-16T03:08:34.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
167172025-12-16T03:08:34.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
167182025-12-16T03:08:34.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
167192025-12-16T03:08:34.325ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
167202025-12-16T03:08:34.325ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.327295ms iteration = 114 status = {"config_updated":false}
167212025-12-16T03:08:39.175ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
167222025-12-16T03:08:39.175ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 20 reason = Timeout
167232025-12-16T03:08:39.176ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
167242025-12-16T03:08:39.176ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
167252025-12-16T03:08:39.176ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
167262025-12-16T03:08:39.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
167272025-12-16T03:08:39.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
167282025-12-16T03:08:39.178ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1526, tv_nsec: 787970333 } })
167292025-12-16T03:08:39.178ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1526, tv_nsec: 788117885 } })
167302025-12-16T03:08:39.178ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
167312025-12-16T03:08:39.179ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1526, tv_nsec: 788576441 } })
167322025-12-16T03:08:39.180ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.168753ms iteration = 20 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
167332025-12-16T03:08:39.238ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
167342025-12-16T03:08:39.240ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
167352025-12-16T03:08:39.240ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
167362025-12-16T03:08:39.240ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 64.73512ms iteration = 20 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
167372025-12-16T03:08:39.240ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5440baab-1944-4322-9c7b-e433c890a79c", "content-length": "26", "date": "Tue, 16 Dec 2025 03:08:39 GMT"} })
167382025-12-16T03:08:39.240ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
167392025-12-16T03:08:39.242ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
167402025-12-16T03:08:39.243ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1526, tv_nsec: 852947857 } })
167412025-12-16T03:08:39.278ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 102.521899ms iteration = 20 status = {}
167422025-12-16T03:08:39.316ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 20 reason = Timeout
167432025-12-16T03:08:39.316ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 39 reason = Timeout
167442025-12-16T03:08:39.316ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
167452025-12-16T03:08:39.316ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 50 reason = Timeout
167462025-12-16T03:08:39.316ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
167472025-12-16T03:08:39.316ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
167482025-12-16T03:08:39.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
167492025-12-16T03:08:39.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
167502025-12-16T03:08:39.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
167512025-12-16T03:08:39.317ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 20 reason = Timeout
167522025-12-16T03:08:39.317ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 12.42\u{b5}s iteration = 20 status = {"disabled":true,"errors":[],"sps":[]}
167532025-12-16T03:08:39.317ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 20 reason = Timeout
167542025-12-16T03:08:39.317ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 20 reason = Timeout
167552025-12-16T03:08:39.317ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
167562025-12-16T03:08:39.317ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
167572025-12-16T03:08:39.317ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 20 reason = Timeout
167582025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
167592025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
167602025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
167612025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
167622025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 39 reason = Timeout
167632025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
167642025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 20 reason = Timeout
167652025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
167662025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 20 reason = Timeout
167672025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
167682025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 32 reason = Timeout
167692025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 30 reason = Timeout
167702025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
167712025-12-16T03:08:39.318ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
167722025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
167732025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 30 reason = Timeout
167742025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
167752025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 58 reason = Timeout
167762025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
167772025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 115 reason = Timeout
167782025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
167792025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 58 reason = Timeout
167802025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
167812025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 20 reason = Timeout
167822025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
167832025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 20 reason = Timeout
167842025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
167852025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 20 reason = Timeout
167862025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
167872025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 20 reason = Timeout
167882025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
167892025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
167902025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
167912025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
167922025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
167932025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
167942025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167952025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
167962025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
167972025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
167982025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
167992025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
168002025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
168012025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
168022025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
168032025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
168042025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
168052025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
168062025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
168072025-12-16T03:08:39.318ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168082025-12-16T03:08:39.319ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168092025-12-16T03:08:39.319ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
168102025-12-16T03:08:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
168112025-12-16T03:08:39.320ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168122025-12-16T03:08:39.320ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168132025-12-16T03:08:39.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168142025-12-16T03:08:39.321ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
168152025-12-16T03:08:39.321ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168162025-12-16T03:08:39.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168172025-12-16T03:08:39.321ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168182025-12-16T03:08:39.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
168192025-12-16T03:08:39.322ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168202025-12-16T03:08:39.322ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
168212025-12-16T03:08:39.322ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
168222025-12-16T03:08:39.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
168232025-12-16T03:08:39.323ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168242025-12-16T03:08:39.324ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
168252025-12-16T03:08:39.325ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
168262025-12-16T03:08:39.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
168272025-12-16T03:08:39.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
168282025-12-16T03:08:39.326ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 7.89217ms iteration = 20 status = {"errors":[],"requests_created_ok":[]}
168292025-12-16T03:08:39.326ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 9.646432ms iteration = 39 status = "NoSitrep"
168302025-12-16T03:08:39.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
168312025-12-16T03:08:39.326ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
168322025-12-16T03:08:39.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
168332025-12-16T03:08:39.326ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
168342025-12-16T03:08:39.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
168352025-12-16T03:08:39.326ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
168362025-12-16T03:08:39.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
168372025-12-16T03:08:39.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
168382025-12-16T03:08:39.327ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
168392025-12-16T03:08:39.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
168402025-12-16T03:08:39.327ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
168412025-12-16T03:08:39.327ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
168422025-12-16T03:08:39.327ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
168432025-12-16T03:08:39.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
168442025-12-16T03:08:39.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
168452025-12-16T03:08:39.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
168462025-12-16T03:08:39.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
168472025-12-16T03:08:39.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
168482025-12-16T03:08:39.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
168492025-12-16T03:08:39.329ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
168502025-12-16T03:08:39.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
168512025-12-16T03:08:39.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
168522025-12-16T03:08:39.330ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1526, tv_nsec: 939683216 } })
168532025-12-16T03:08:39.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
168542025-12-16T03:08:39.330ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
168552025-12-16T03:08:39.330ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
168562025-12-16T03:08:39.331ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
168572025-12-16T03:08:39.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
168582025-12-16T03:08:39.331ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
168592025-12-16T03:08:39.331ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
168602025-12-16T03:08:39.331ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1526, tv_nsec: 940640278 } })
168612025-12-16T03:08:39.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
168622025-12-16T03:08:39.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
168632025-12-16T03:08:39.331ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 1845c551-ac2c-40d7-ad4c-fd64ea891f40 original_time_started = 2025-12-16 03:08:10.631572 UTC
168642025-12-16T03:08:39.331ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 15.381925ms iteration = 50 status = {"Loaded":{"collection_id":"1845c551-ac2c-40d7-ad4c-fd64ea891f40","time_loaded":"2025-12-16T03:08:39.316272377Z","time_started":"2025-12-16T03:08:10.631572Z"}}
168652025-12-16T03:08:39.331ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
168662025-12-16T03:08:39.331ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168672025-12-16T03:08:39.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
168682025-12-16T03:08:39.332ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
168692025-12-16T03:08:39.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
168702025-12-16T03:08:39.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
168712025-12-16T03:08:39.333ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
168722025-12-16T03:08:39.333ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 14.963229ms iteration = 115 status = {"config_updated":false}
168732025-12-16T03:08:39.333ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168742025-12-16T03:08:39.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
168752025-12-16T03:08:39.333ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
168762025-12-16T03:08:39.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
168772025-12-16T03:08:39.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168782025-12-16T03:08:39.335ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168792025-12-16T03:08:39.335ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 17.202798ms iteration = 20 status = 0
168802025-12-16T03:08:39.335ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
168812025-12-16T03:08:39.335ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
168822025-12-16T03:08:39.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
168832025-12-16T03:08:39.335ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168842025-12-16T03:08:39.336ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
168852025-12-16T03:08:39.336ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
168862025-12-16T03:08:39.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
168872025-12-16T03:08:39.337ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
168882025-12-16T03:08:39.337ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
168892025-12-16T03:08:39.337ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 19.978834ms iteration = 39 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
168902025-12-16T03:08:39.340ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
168912025-12-16T03:08:39.340ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
168922025-12-16T03:08:39.340ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
168932025-12-16T03:08:39.340ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
168942025-12-16T03:08:39.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
168952025-12-16T03:08:39.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
168962025-12-16T03:08:39.368ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
168972025-12-16T03:08:39.368ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 50.039304ms iteration = 58 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
168982025-12-16T03:08:39.372ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 54.44186ms iteration = 20 status = {"errors":[],"finish_invoked_ok":[]}
168992025-12-16T03:08:39.373ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
169002025-12-16T03:08:39.373ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
169012025-12-16T03:08:39.373ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
169022025-12-16T03:08:39.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
169032025-12-16T03:08:39.374ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "801c5164-68be-4304-8cf5-d44d656addde", "content-length": "26", "date": "Tue, 16 Dec 2025 03:08:39 GMT"} })
169042025-12-16T03:08:39.374ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
169052025-12-16T03:08:39.375ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
169062025-12-16T03:08:39.375ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 57.830563ms iteration = 20 status = {"errors":[],"garbage_collect_requested":[]}
169072025-12-16T03:08:39.375ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 57.750882ms iteration = 58 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
169082025-12-16T03:08:39.376ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
169092025-12-16T03:08:39.376ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
169102025-12-16T03:08:39.376ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
169112025-12-16T03:08:39.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
169122025-12-16T03:08:39.380ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
169132025-12-16T03:08:39.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
169142025-12-16T03:08:39.382ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
169152025-12-16T03:08:39.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
169162025-12-16T03:08:39.384ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
169172025-12-16T03:08:39.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
169182025-12-16T03:08:39.390ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
169192025-12-16T03:08:39.390ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 72.163885ms iteration = 30 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
169202025-12-16T03:08:39.390ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
169212025-12-16T03:08:39.390ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 72.527428ms iteration = 20 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
169222025-12-16T03:08:39.393ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 75.686689ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
169232025-12-16T03:08:39.394ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 76.181285ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
169242025-12-16T03:08:39.409ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1527, tv_nsec: 19370085 } })
169252025-12-16T03:08:39.423ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
169262025-12-16T03:08:39.423ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
169272025-12-16T03:08:39.423ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
169282025-12-16T03:08:39.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
169292025-12-16T03:08:39.426ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 108.594766ms iteration = 20 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
169302025-12-16T03:08:39.441ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1527, tv_nsec: 50458509 } })
169312025-12-16T03:08:39.454ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 136.877104ms iteration = 32 status = {"physical_disks_added":0}
169322025-12-16T03:08:39.464ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
169332025-12-16T03:08:39.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
169342025-12-16T03:08:39.466ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169352025-12-16T03:08:39.466ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
169362025-12-16T03:08:39.466ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
169372025-12-16T03:08:39.466ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 148.885036ms iteration = 20 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
169382025-12-16T03:08:39.468ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
169392025-12-16T03:08:39.468ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
169402025-12-16T03:08:39.469ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
169412025-12-16T03:08:39.470ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169422025-12-16T03:08:39.472ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
169432025-12-16T03:08:39.472ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
169442025-12-16T03:08:39.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
169452025-12-16T03:08:39.478ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
169462025-12-16T03:08:39.478ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
169472025-12-16T03:08:39.478ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
169482025-12-16T03:08:39.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
169492025-12-16T03:08:39.479ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "24687eb2-0ed1-4639-a038-abb2b3190383", "content-length": "372", "date": "Tue, 16 Dec 2025 03:08:39 GMT"} })
169502025-12-16T03:08:39.479ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
169512025-12-16T03:08:39.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
169522025-12-16T03:08:39.480ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
169532025-12-16T03:08:39.480ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
169542025-12-16T03:08:39.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
169552025-12-16T03:08:39.494ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
169562025-12-16T03:08:39.494ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
169572025-12-16T03:08:39.494ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
169582025-12-16T03:08:39.494ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
169592025-12-16T03:08:39.494ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
169602025-12-16T03:08:39.494ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
169612025-12-16T03:08:39.494ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
169622025-12-16T03:08:39.494ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
169632025-12-16T03:08:39.495ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
169642025-12-16T03:08:39.495ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
169652025-12-16T03:08:39.495ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
169662025-12-16T03:08:39.495ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
169672025-12-16T03:08:39.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
169682025-12-16T03:08:39.496ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
169692025-12-16T03:08:39.496ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
169702025-12-16T03:08:39.497ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "1084fb59-1934-45df-b65c-bb5c89f62cce", "date": "Tue, 16 Dec 2025 03:08:39 GMT"} })
169712025-12-16T03:08:39.497ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
169722025-12-16T03:08:39.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
169732025-12-16T03:08:39.498ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
169742025-12-16T03:08:39.498ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
169752025-12-16T03:08:39.499ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
169762025-12-16T03:08:39.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
169772025-12-16T03:08:39.502ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
169782025-12-16T03:08:39.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
169792025-12-16T03:08:39.505ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
169802025-12-16T03:08:39.505ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
169812025-12-16T03:08:39.506ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "b02be6d1-6dd4-4b94-a961-42e66aeffa34", "date": "Tue, 16 Dec 2025 03:08:39 GMT"} })
169822025-12-16T03:08:39.508ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 190.302961ms iteration = 30 status = {}
169832025-12-16T03:08:39.508ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
169842025-12-16T03:08:39.509ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a114781-7817-4e44-b0e2-630aec1c5a62", "content-length": "691", "date": "Tue, 16 Dec 2025 03:08:39 GMT"} })
169852025-12-16T03:08:39.509ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
169862025-12-16T03:08:39.509ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
169872025-12-16T03:08:39.509ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
169882025-12-16T03:08:39.510ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 192.622289ms iteration = 20 status = {}
169892025-12-16T03:08:39.704ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "690b2bc9-a764-4537-aaae-7276fcc0e30f", "date": "Tue, 16 Dec 2025 03:08:39 GMT"} })
169902025-12-16T03:08:39.704ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
169912025-12-16T03:08:39.704ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
169922025-12-16T03:08:39.704ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d746754a-2592-421f-993d-5cf67c145909", "content-length": "682", "date": "Tue, 16 Dec 2025 03:08:39 GMT"} })
169932025-12-16T03:08:39.705ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
169942025-12-16T03:08:39.705ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
169952025-12-16T03:08:39.708ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
169962025-12-16T03:08:39.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
169972025-12-16T03:08:39.711ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
169982025-12-16T03:08:39.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
169992025-12-16T03:08:39.712ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
170002025-12-16T03:08:39.713ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
170012025-12-16T03:08:39.720ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 404.099519ms iteration = 20 status = {}
170022025-12-16T03:08:41.068ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 290 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = bdf949ef-b5e1-43ed-ae05-a241b1a82629 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
170032025-12-16T03:08:44.317ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 116 reason = Timeout
170042025-12-16T03:08:44.317ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
170052025-12-16T03:08:44.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
170062025-12-16T03:08:44.320ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170072025-12-16T03:08:44.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
170082025-12-16T03:08:44.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
170092025-12-16T03:08:44.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
170102025-12-16T03:08:44.325ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
170112025-12-16T03:08:44.325ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.250674ms iteration = 116 status = {"config_updated":false}
170122025-12-16T03:08:49.317ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 59 reason = Timeout
170132025-12-16T03:08:49.317ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 59 reason = Timeout
170142025-12-16T03:08:49.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
170152025-12-16T03:08:49.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
170162025-12-16T03:08:49.317ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 117 reason = Timeout
170172025-12-16T03:08:49.317ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
170182025-12-16T03:08:49.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
170192025-12-16T03:08:49.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
170202025-12-16T03:08:49.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
170212025-12-16T03:08:49.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
170222025-12-16T03:08:49.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170232025-12-16T03:08:49.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170242025-12-16T03:08:49.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
170252025-12-16T03:08:49.321ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.777058ms iteration = 59 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
170262025-12-16T03:08:49.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
170272025-12-16T03:08:49.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
170282025-12-16T03:08:49.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
170292025-12-16T03:08:49.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
170302025-12-16T03:08:49.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
170312025-12-16T03:08:49.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
170322025-12-16T03:08:49.325ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
170332025-12-16T03:08:49.325ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.124553ms iteration = 117 status = {"config_updated":false}
170342025-12-16T03:08:49.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
170352025-12-16T03:08:49.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
170362025-12-16T03:08:49.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170372025-12-16T03:08:49.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
170382025-12-16T03:08:49.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
170392025-12-16T03:08:49.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
170402025-12-16T03:08:49.343ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
170412025-12-16T03:08:49.343ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.787057ms iteration = 59 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
170422025-12-16T03:08:51.067ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 280 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 3a14674c-1ede-413b-b577-fc7bfa825811 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
170432025-12-16T03:08:54.316ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 40 reason = Timeout
170442025-12-16T03:08:54.317ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 51 reason = Timeout
170452025-12-16T03:08:54.317ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
170462025-12-16T03:08:54.317ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
170472025-12-16T03:08:54.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
170482025-12-16T03:08:54.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
170492025-12-16T03:08:54.317ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 118 reason = Timeout
170502025-12-16T03:08:54.317ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 40 reason = Timeout
170512025-12-16T03:08:54.317ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
170522025-12-16T03:08:54.317ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
170532025-12-16T03:08:54.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
170542025-12-16T03:08:54.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
170552025-12-16T03:08:54.318ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
170562025-12-16T03:08:54.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
170572025-12-16T03:08:54.319ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170582025-12-16T03:08:54.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170592025-12-16T03:08:54.319ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170602025-12-16T03:08:54.320ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170612025-12-16T03:08:54.322ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
170622025-12-16T03:08:54.323ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
170632025-12-16T03:08:54.323ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
170642025-12-16T03:08:54.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
170652025-12-16T03:08:54.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
170662025-12-16T03:08:54.323ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
170672025-12-16T03:08:54.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
170682025-12-16T03:08:54.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
170692025-12-16T03:08:54.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
170702025-12-16T03:08:54.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
170712025-12-16T03:08:54.324ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
170722025-12-16T03:08:54.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
170732025-12-16T03:08:54.324ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 7.993701ms iteration = 40 status = "NoSitrep"
170742025-12-16T03:08:54.325ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
170752025-12-16T03:08:54.325ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 1845c551-ac2c-40d7-ad4c-fd64ea891f40 original_time_started = 2025-12-16 03:08:10.631572 UTC
170762025-12-16T03:08:54.325ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.502918ms iteration = 51 status = {"Loaded":{"collection_id":"1845c551-ac2c-40d7-ad4c-fd64ea891f40","time_loaded":"2025-12-16T03:08:54.316920922Z","time_started":"2025-12-16T03:08:10.631572Z"}}
170772025-12-16T03:08:54.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
170782025-12-16T03:08:54.325ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
170792025-12-16T03:08:54.325ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.317295ms iteration = 118 status = {"config_updated":false}
170802025-12-16T03:08:54.327ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170812025-12-16T03:08:54.328ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
170822025-12-16T03:08:54.329ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
170832025-12-16T03:08:54.329ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 11.654787ms iteration = 40 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
170842025-12-16T03:08:59.317ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 60 reason = Timeout
170852025-12-16T03:08:59.317ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
170862025-12-16T03:08:59.317ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 60 reason = Timeout
170872025-12-16T03:08:59.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
170882025-12-16T03:08:59.317ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 119 reason = Timeout
170892025-12-16T03:08:59.317ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
170902025-12-16T03:08:59.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
170912025-12-16T03:08:59.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
170922025-12-16T03:08:59.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
170932025-12-16T03:08:59.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
170942025-12-16T03:08:59.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170952025-12-16T03:08:59.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170962025-12-16T03:08:59.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
170972025-12-16T03:08:59.322ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.624078ms iteration = 60 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
170982025-12-16T03:08:59.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
170992025-12-16T03:08:59.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
171002025-12-16T03:08:59.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
171012025-12-16T03:08:59.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
171022025-12-16T03:08:59.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
171032025-12-16T03:08:59.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
171042025-12-16T03:08:59.325ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
171052025-12-16T03:08:59.325ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.085182ms iteration = 119 status = {"config_updated":false}
171062025-12-16T03:08:59.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
171072025-12-16T03:08:59.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
171082025-12-16T03:08:59.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171092025-12-16T03:08:59.331ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
171102025-12-16T03:08:59.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
171112025-12-16T03:08:59.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
171122025-12-16T03:08:59.348ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
171132025-12-16T03:08:59.348ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 31.262006ms iteration = 60 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
171142025-12-16T03:09:01.068ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 453 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = c7c01ec7-6d85-46e8-8a45-82fc554e7c9b response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
171152025-12-16T03:09:04.317ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 120 reason = Timeout
171162025-12-16T03:09:04.317ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
171172025-12-16T03:09:04.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
171182025-12-16T03:09:04.320ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171192025-12-16T03:09:04.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
171202025-12-16T03:09:04.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
171212025-12-16T03:09:04.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
171222025-12-16T03:09:04.325ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
171232025-12-16T03:09:04.325ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.964651ms iteration = 120 status = {"config_updated":false}
171242025-12-16T03:09:06.061ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:65175
171252025-12-16T03:09:06.061ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:65175 req_id = d570ced7-64e6-4f57-b0d6-ef043fcca216 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending version = 1.0.0
171262025-12-16T03:09:06.061ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:65175 req_id = d570ced7-64e6-4f57-b0d6-ef043fcca216 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
171272025-12-16T03:09:06.062ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:65175 req_id = d570ced7-64e6-4f57-b0d6-ef043fcca216 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
171282025-12-16T03:09:06.064ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 3375 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:65175 req_id = d570ced7-64e6-4f57-b0d6-ef043fcca216 response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending
171292025-12-16T03:09:06.065ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:65175 req_id = 6d975bf6-2fc8-4a04-a97a-3b79e07fab55 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D version = 1.0.0
171302025-12-16T03:09:06.065ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:65175 req_id = 6d975bf6-2fc8-4a04-a97a-3b79e07fab55 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
171312025-12-16T03:09:06.065ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:65175 req_id = 6d975bf6-2fc8-4a04-a97a-3b79e07fab55 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
171322025-12-16T03:09:06.067ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 2654 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:65175 req_id = 6d975bf6-2fc8-4a04-a97a-3b79e07fab55 response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D
171332025-12-16T03:09:09.175ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
171342025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
171352025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
171362025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
171372025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
171382025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
171392025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
171402025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
171412025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
171422025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
171432025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
171442025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 11 reason = Timeout
171452025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 21 reason = Timeout
171462025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
171472025-12-16T03:09:09.176ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-12-16 02:59:09.176106995 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
171482025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
171492025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
171502025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 21 reason = Timeout
171512025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
171522025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
171532025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 21 reason = Timeout
171542025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
171552025-12-16T03:09:09.176ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
171562025-12-16T03:09:09.176ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 00095469-1d05-4e68-8de6-a3957b71e34b step component = ExternalNetworking step id = Ensure
171572025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171582025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
171592025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
171602025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
171612025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171622025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
171632025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
171642025-12-16T03:09:09.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
171652025-12-16T03:09:09.176ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1a45cd23-3d0a-43eb-adce-62c7b712baf1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1556, tv_nsec: 785928927 } })
171662025-12-16T03:09:09.177ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("49a51a59-9919-48ae-8182-4070a8430160.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6af60386-8771-4401-9d4d-e93774e519f0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 1556, tv_nsec: 785954818 } })
171672025-12-16T03:09:09.177ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
171682025-12-16T03:09:09.177ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.648021ms iteration = 11 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
171692025-12-16T03:09:09.177ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
171702025-12-16T03:09:09.177ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.711591ms iteration = 11 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
171712025-12-16T03:09:09.178ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
171722025-12-16T03:09:09.178ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
171732025-12-16T03:09:09.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
171742025-12-16T03:09:09.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
171752025-12-16T03:09:09.179ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1556, tv_nsec: 788367148 } })
171762025-12-16T03:09:09.181ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171772025-12-16T03:09:09.181ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1556, tv_nsec: 790807879 } })
171782025-12-16T03:09:09.182ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
171792025-12-16T03:09:09.182ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171802025-12-16T03:09:09.182ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1556, tv_nsec: 791584279 } })
171812025-12-16T03:09:09.183ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171822025-12-16T03:09:09.183ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-12-16 02:59:09.176106995 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
171832025-12-16T03:09:09.183ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 7.383243ms iteration = 11 status = {"expiration":"2025-12-16T02:59:09.176106995Z","pruned":[]}
171842025-12-16T03:09:09.186ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
171852025-12-16T03:09:09.186ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
171862025-12-16T03:09:09.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
171872025-12-16T03:09:09.190ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
171882025-12-16T03:09:09.190ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
171892025-12-16T03:09:09.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171902025-12-16T03:09:09.193ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
171912025-12-16T03:09:09.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
171922025-12-16T03:09:09.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
171932025-12-16T03:09:09.195ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 18.94687ms iteration = 21 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
171942025-12-16T03:09:09.197ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
171952025-12-16T03:09:09.197ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
171962025-12-16T03:09:09.197ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
171972025-12-16T03:09:09.197ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
171982025-12-16T03:09:09.197ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
171992025-12-16T03:09:09.197ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172002025-12-16T03:09:09.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
172012025-12-16T03:09:09.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
172022025-12-16T03:09:09.199ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172032025-12-16T03:09:09.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172042025-12-16T03:09:09.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
172052025-12-16T03:09:09.203ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
172062025-12-16T03:09:09.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
172072025-12-16T03:09:09.205ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
172082025-12-16T03:09:09.205ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
172092025-12-16T03:09:09.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
172102025-12-16T03:09:09.207ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
172112025-12-16T03:09:09.209ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
172122025-12-16T03:09:09.209ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
172132025-12-16T03:09:09.209ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
172142025-12-16T03:09:09.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
172152025-12-16T03:09:09.210ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
172162025-12-16T03:09:09.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
172172025-12-16T03:09:09.211ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
172182025-12-16T03:09:09.212ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172192025-12-16T03:09:09.213ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172202025-12-16T03:09:09.214ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
172212025-12-16T03:09:09.217ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
172222025-12-16T03:09:09.217ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 12 reason = Timeout
172232025-12-16T03:09:09.217ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 25 reason = Timeout
172242025-12-16T03:09:09.217ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
172252025-12-16T03:09:09.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
172262025-12-16T03:09:09.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
172272025-12-16T03:09:09.218ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 15 reason = Timeout
172282025-12-16T03:09:09.218ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 11 reason = Timeout
172292025-12-16T03:09:09.218ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
172302025-12-16T03:09:09.218ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 11 reason = Timeout
172312025-12-16T03:09:09.218ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
172322025-12-16T03:09:09.218ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
172332025-12-16T03:09:09.218ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 3 reason = Timeout
172342025-12-16T03:09:09.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
172352025-12-16T03:09:09.218ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
172362025-12-16T03:09:09.218ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
172372025-12-16T03:09:09.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
172382025-12-16T03:09:09.218ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 11 reason = Timeout
172392025-12-16T03:09:09.218ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
172402025-12-16T03:09:09.218ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
172412025-12-16T03:09:09.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
172422025-12-16T03:09:09.218ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
172432025-12-16T03:09:09.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
172442025-12-16T03:09:09.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
172452025-12-16T03:09:09.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
172462025-12-16T03:09:09.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
172472025-12-16T03:09:09.218ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
172482025-12-16T03:09:09.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
172492025-12-16T03:09:09.219ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
172502025-12-16T03:09:09.219ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
172512025-12-16T03:09:09.220ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172522025-12-16T03:09:09.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
172532025-12-16T03:09:09.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172542025-12-16T03:09:09.220ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172552025-12-16T03:09:09.221ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
172562025-12-16T03:09:09.221ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
172572025-12-16T03:09:09.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
172582025-12-16T03:09:09.223ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
172592025-12-16T03:09:09.224ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
172602025-12-16T03:09:09.224ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
172612025-12-16T03:09:09.224ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 6.163029ms iteration = 11 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
172622025-12-16T03:09:09.224ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
172632025-12-16T03:09:09.224ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
172642025-12-16T03:09:09.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
172652025-12-16T03:09:09.224ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
172662025-12-16T03:09:09.224ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
172672025-12-16T03:09:09.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
172682025-12-16T03:09:09.225ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
172692025-12-16T03:09:09.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
172702025-12-16T03:09:09.225ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
172712025-12-16T03:09:09.226ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172722025-12-16T03:09:09.226ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
172732025-12-16T03:09:09.226ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
172742025-12-16T03:09:09.227ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
172752025-12-16T03:09:09.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
172762025-12-16T03:09:09.228ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
172772025-12-16T03:09:09.228ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172782025-12-16T03:09:09.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
172792025-12-16T03:09:09.229ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
172802025-12-16T03:09:09.230ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
172812025-12-16T03:09:09.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
172822025-12-16T03:09:09.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
172832025-12-16T03:09:09.231ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
172842025-12-16T03:09:09.231ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
172852025-12-16T03:09:09.231ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
172862025-12-16T03:09:09.231ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
172872025-12-16T03:09:09.232ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-12-16 02:59:16.791641 UTC dns_group = internal generation = 1
172882025-12-16T03:09:09.232ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 55.978969ms iteration = 11 status = {"generation":1}
172892025-12-16T03:09:09.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
172902025-12-16T03:09:09.232ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
172912025-12-16T03:09:09.232ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
172922025-12-16T03:09:09.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
172932025-12-16T03:09:09.233ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172942025-12-16T03:09:09.234ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
172952025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 2 reason = Timeout
172962025-12-16T03:09:09.238ZINFOnexus (SagaQuiesceHandle): saga recovery pass starting file = nexus/types/src/quiesce.rs:510
172972025-12-16T03:09:09.238ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
172982025-12-16T03:09:09.238ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
172992025-12-16T03:09:09.238ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a (built_in_user), .. }) resource = Database result = Ok(())
173002025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): activating background_task = tuf_repo_pruner iteration = 4 reason = Timeout
173012025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
173022025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
173032025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 3 reason = Timeout
173042025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
173052025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
173062025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 34 reason = Timeout
173072025-12-16T03:09:09.238ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
173082025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 14.85\u{b5}s iteration = 34 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
173092025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 11 reason = Timeout
173102025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
173112025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
173122025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
173132025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
173142025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 11 reason = Timeout
173152025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
173162025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
173172025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
173182025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 11 reason = Timeout
173192025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 11 reason = Timeout
173202025-12-16T03:09:09.238ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
173212025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
173222025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
173232025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
173242025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
173252025-12-16T03:09:09.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
173262025-12-16T03:09:09.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
173272025-12-16T03:09:09.239ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
173282025-12-16T03:09:09.240ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 00095469-1d05-4e68-8de6-a3957b71e34b step component = SledList step id = Fetch
173292025-12-16T03:09:09.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
173302025-12-16T03:09:09.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
173312025-12-16T03:09:09.241ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173322025-12-16T03:09:09.241ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173332025-12-16T03:09:09.242ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.968873ms iteration = 11 status = {"errors":[],"found_port_ok":[]}
173342025-12-16T03:09:09.245ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
173352025-12-16T03:09:09.245ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
173362025-12-16T03:09:09.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
173372025-12-16T03:09:09.248ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173382025-12-16T03:09:09.249ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
173392025-12-16T03:09:09.249ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
173402025-12-16T03:09:09.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
173412025-12-16T03:09:09.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
173422025-12-16T03:09:09.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
173432025-12-16T03:09:09.253ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
173442025-12-16T03:09:09.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
173452025-12-16T03:09:09.256ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
173462025-12-16T03:09:09.256ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 18.800828ms iteration = 11 status = {"by_rx":{},"error":null}
173472025-12-16T03:09:09.257ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173482025-12-16T03:09:09.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173492025-12-16T03:09:09.257ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173502025-12-16T03:09:09.258ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
173512025-12-16T03:09:09.258ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
173522025-12-16T03:09:09.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
173532025-12-16T03:09:09.260ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
173542025-12-16T03:09:09.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
173552025-12-16T03:09:09.262ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Fleet result = Ok(())
173562025-12-16T03:09:09.262ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
173572025-12-16T03:09:09.262ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
173582025-12-16T03:09:09.262ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
173592025-12-16T03:09:09.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
173602025-12-16T03:09:09.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
173612025-12-16T03:09:09.262ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
173622025-12-16T03:09:09.262ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
173632025-12-16T03:09:09.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
173642025-12-16T03:09:09.272ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
173652025-12-16T03:09:09.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173662025-12-16T03:09:09.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
173672025-12-16T03:09:09.274ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
173682025-12-16T03:09:09.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
173692025-12-16T03:09:09.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173702025-12-16T03:09:09.276ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173712025-12-16T03:09:09.276ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173722025-12-16T03:09:09.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173732025-12-16T03:09:09.278ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
173742025-12-16T03:09:09.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
173752025-12-16T03:09:09.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
173762025-12-16T03:09:09.279ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
173772025-12-16T03:09:09.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
173782025-12-16T03:09:09.281ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173792025-12-16T03:09:09.282ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
173802025-12-16T03:09:09.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
173812025-12-16T03:09:09.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
173822025-12-16T03:09:09.282ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
173832025-12-16T03:09:09.282ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
173842025-12-16T03:09:09.282ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
173852025-12-16T03:09:09.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
173862025-12-16T03:09:09.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
173872025-12-16T03:09:09.282ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Fleet result = Ok(())
173882025-12-16T03:09:09.283ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
173892025-12-16T03:09:09.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
173902025-12-16T03:09:09.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
173912025-12-16T03:09:09.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173922025-12-16T03:09:09.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173932025-12-16T03:09:09.286ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
173942025-12-16T03:09:09.287ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 48.702457ms iteration = 11 status = {"dispatched":[],"errors":[],"glob_version":"212.0.0","globs_reprocessed":{},"no_receivers":[]}
173952025-12-16T03:09:09.287ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
173962025-12-16T03:09:09.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
173972025-12-16T03:09:09.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
173982025-12-16T03:09:09.288ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 00095469-1d05-4e68-8de6-a3957b71e34b step component = DeployNexusRecords step id = Ensure
173992025-12-16T03:09:09.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
174002025-12-16T03:09:09.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
174012025-12-16T03:09:09.289ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174022025-12-16T03:09:09.289ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174032025-12-16T03:09:09.290ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
174042025-12-16T03:09:09.290ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
174052025-12-16T03:09:09.290ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
174062025-12-16T03:09:09.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174072025-12-16T03:09:09.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
174082025-12-16T03:09:09.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174092025-12-16T03:09:09.291ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
174102025-12-16T03:09:09.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
174112025-12-16T03:09:09.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174122025-12-16T03:09:09.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
174132025-12-16T03:09:09.295ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
174142025-12-16T03:09:09.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
174152025-12-16T03:09:09.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
174162025-12-16T03:09:09.295ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
174172025-12-16T03:09:09.295ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
174182025-12-16T03:09:09.295ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
174192025-12-16T03:09:09.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
174202025-12-16T03:09:09.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
174212025-12-16T03:09:09.297ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
174222025-12-16T03:09:09.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
174232025-12-16T03:09:09.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
174242025-12-16T03:09:09.297ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = TargetReleaseConfig result = Ok(())
174252025-12-16T03:09:09.297ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
174262025-12-16T03:09:09.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
174272025-12-16T03:09:09.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174282025-12-16T03:09:09.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174292025-12-16T03:09:09.299ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
174302025-12-16T03:09:09.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174312025-12-16T03:09:09.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
174322025-12-16T03:09:09.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
174332025-12-16T03:09:09.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
174342025-12-16T03:09:09.302ZINFOnexus (BackgroundTasks): tuf_repo_prune decision background_task = tuf_repo_pruner file = nexus/src/app/background/tasks/tuf_repo_pruner.rs:153 status = TufRepoPrunerDetails { nkeep_recent_releases: 3, nkeep_recent_uploads: 2, repos_keep_target_release: {}, repos_keep_recent_uploads: {}, repo_prune: None, other_repos_eligible_to_prune: {}, warnings: [] }
174352025-12-16T03:09:09.302ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_repo_pruner elapsed = 66.494752ms iteration = 4 status = {"nkeep_recent_releases":3,"nkeep_recent_uploads":2,"other_repos_eligible_to_prune":[],"repo_prune":null,"repos_keep_recent_uploads":[],"repos_keep_target_release":[],"status":"enabled","warnings":[]}
174362025-12-16T03:09:09.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174372025-12-16T03:09:09.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174382025-12-16T03:09:09.303ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
174392025-12-16T03:09:09.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
174402025-12-16T03:09:09.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
174412025-12-16T03:09:09.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174422025-12-16T03:09:09.305ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
174432025-12-16T03:09:09.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
174442025-12-16T03:09:09.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174452025-12-16T03:09:09.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174462025-12-16T03:09:09.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
174472025-12-16T03:09:09.309ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
174482025-12-16T03:09:09.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
174492025-12-16T03:09:09.311ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
174502025-12-16T03:09:09.311ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
174512025-12-16T03:09:09.312ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
174522025-12-16T03:09:09.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
174532025-12-16T03:09:09.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
174542025-12-16T03:09:09.313ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
174552025-12-16T03:09:09.313ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
174562025-12-16T03:09:09.313ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
174572025-12-16T03:09:09.313ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
174582025-12-16T03:09:09.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
174592025-12-16T03:09:09.314ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
174602025-12-16T03:09:09.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
174612025-12-16T03:09:09.314ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
174622025-12-16T03:09:09.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
174632025-12-16T03:09:09.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
174642025-12-16T03:09:09.317ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 52 reason = Timeout
174652025-12-16T03:09:09.317ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 41 reason = Timeout
174662025-12-16T03:09:09.317ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
174672025-12-16T03:09:09.317ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
174682025-12-16T03:09:09.317ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 21 reason = Timeout
174692025-12-16T03:09:09.317ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
174702025-12-16T03:09:09.317ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
174712025-12-16T03:09:09.317ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174722025-12-16T03:09:09.317ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
174732025-12-16T03:09:09.317ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 99.4008ms iteration = 15 status = {"blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","inventory_collection_id":"1845c551-ac2c-40d7-ad4c-fd64ea891f40","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
174742025-12-16T03:09:09.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
174752025-12-16T03:09:09.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
174762025-12-16T03:09:09.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
174772025-12-16T03:09:09.318ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 21 reason = Timeout
174782025-12-16T03:09:09.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 21 reason = Timeout
174792025-12-16T03:09:09.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
174802025-12-16T03:09:09.318ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 23.17\u{b5}s iteration = 21 status = {"disabled":true,"errors":[],"sps":[]}
174812025-12-16T03:09:09.318ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174822025-12-16T03:09:09.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 21 reason = Timeout
174832025-12-16T03:09:09.318ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 41 reason = Timeout
174842025-12-16T03:09:09.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
174852025-12-16T03:09:09.318ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
174862025-12-16T03:09:09.318ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 21 reason = Timeout
174872025-12-16T03:09:09.318ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 21 reason = Timeout
174882025-12-16T03:09:09.318ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
174892025-12-16T03:09:09.318ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
174902025-12-16T03:09:09.318ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
174912025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 61 reason = Timeout
174922025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
174932025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 31 reason = Timeout
174942025-12-16T03:09:09.319ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
174952025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
174962025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 21 reason = Timeout
174972025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
174982025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 61 reason = Timeout
174992025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
175002025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 121 reason = Timeout
175012025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
175022025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
175032025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 21 reason = Timeout
175042025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 21 reason = Timeout
175052025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
175062025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
175072025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 33 reason = Timeout
175082025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
175092025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
175102025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 21 reason = Timeout
175112025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
175122025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 21 reason = Timeout
175132025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 21 reason = Timeout
175142025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
175152025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
175162025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 31 reason = Timeout
175172025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
175182025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
175192025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
175202025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
175212025-12-16T03:09:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
175222025-12-16T03:09:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
175232025-12-16T03:09:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
175242025-12-16T03:09:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
175252025-12-16T03:09:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
175262025-12-16T03:09:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
175272025-12-16T03:09:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
175282025-12-16T03:09:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
175292025-12-16T03:09:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
175302025-12-16T03:09:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
175312025-12-16T03:09:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
175322025-12-16T03:09:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
175332025-12-16T03:09:09.320ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175342025-12-16T03:09:09.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175352025-12-16T03:09:09.321ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175362025-12-16T03:09:09.321ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
175372025-12-16T03:09:09.321ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
175382025-12-16T03:09:09.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
175392025-12-16T03:09:09.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
175402025-12-16T03:09:09.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
175412025-12-16T03:09:09.321ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7c1b3f0-0837-4a97-9d4d-55cdf4366342", "content-length": "26", "date": "Tue, 16 Dec 2025 03:09:09 GMT"} })
175422025-12-16T03:09:09.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
175432025-12-16T03:09:09.321ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
175442025-12-16T03:09:09.323ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175452025-12-16T03:09:09.323ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175462025-12-16T03:09:09.324ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175472025-12-16T03:09:09.325ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175482025-12-16T03:09:09.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
175492025-12-16T03:09:09.327ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
175502025-12-16T03:09:09.327ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
175512025-12-16T03:09:09.327ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
175522025-12-16T03:09:09.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
175532025-12-16T03:09:09.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
175542025-12-16T03:09:09.328ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175552025-12-16T03:09:09.328ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175562025-12-16T03:09:09.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
175572025-12-16T03:09:09.328ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
175582025-12-16T03:09:09.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
175592025-12-16T03:09:09.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
175602025-12-16T03:09:09.329ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
175612025-12-16T03:09:09.330ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
175622025-12-16T03:09:09.330ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
175632025-12-16T03:09:09.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
175642025-12-16T03:09:09.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
175652025-12-16T03:09:09.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
175662025-12-16T03:09:09.330ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
175672025-12-16T03:09:09.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
175682025-12-16T03:09:09.331ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
175692025-12-16T03:09:09.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
175702025-12-16T03:09:09.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
175712025-12-16T03:09:09.331ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
175722025-12-16T03:09:09.331ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
175732025-12-16T03:09:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
175742025-12-16T03:09:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
175752025-12-16T03:09:09.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175762025-12-16T03:09:09.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
175772025-12-16T03:09:09.334ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
175782025-12-16T03:09:09.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
175792025-12-16T03:09:09.335ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
175802025-12-16T03:09:09.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
175812025-12-16T03:09:09.336ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175822025-12-16T03:09:09.336ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 17.495212ms iteration = 21 status = {"errors":[],"garbage_collect_requested":[]}
175832025-12-16T03:09:09.336ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
175842025-12-16T03:09:09.336ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
175852025-12-16T03:09:09.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
175862025-12-16T03:09:09.336ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
175872025-12-16T03:09:09.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
175882025-12-16T03:09:09.337ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 20.908295ms iteration = 41 status = "NoSitrep"
175892025-12-16T03:09:09.338ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 1845c551-ac2c-40d7-ad4c-fd64ea891f40 original_time_started = 2025-12-16 03:08:10.631572 UTC
175902025-12-16T03:09:09.338ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 21.886857ms iteration = 52 status = {"Loaded":{"collection_id":"1845c551-ac2c-40d7-ad4c-fd64ea891f40","time_loaded":"2025-12-16T03:09:09.317068941Z","time_started":"2025-12-16T03:08:10.631572Z"}}
175912025-12-16T03:09:09.339ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
175922025-12-16T03:09:09.339ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
175932025-12-16T03:09:09.339ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
175942025-12-16T03:09:09.339ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
175952025-12-16T03:09:09.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
175962025-12-16T03:09:09.339ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 163.527262ms iteration = 21 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
175972025-12-16T03:09:09.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
175982025-12-16T03:09:09.340ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
175992025-12-16T03:09:09.340ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
176002025-12-16T03:09:09.340ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176012025-12-16T03:09:09.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
176022025-12-16T03:09:09.341ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1556, tv_nsec: 950224149 } })
176032025-12-16T03:09:09.341ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
176042025-12-16T03:09:09.342ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
176052025-12-16T03:09:09.343ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
176062025-12-16T03:09:09.343ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
176072025-12-16T03:09:09.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
176082025-12-16T03:09:09.343ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1556, tv_nsec: 952090133 } })
176092025-12-16T03:09:09.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
176102025-12-16T03:09:09.344ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
176112025-12-16T03:09:09.345ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176122025-12-16T03:09:09.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
176132025-12-16T03:09:09.345ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176142025-12-16T03:09:09.345ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
176152025-12-16T03:09:09.345ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
176162025-12-16T03:09:09.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
176172025-12-16T03:09:09.346ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
176182025-12-16T03:09:09.346ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 27.194415ms iteration = 121 status = {"config_updated":false}
176192025-12-16T03:09:09.346ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
176202025-12-16T03:09:09.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
176212025-12-16T03:09:09.351ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
176222025-12-16T03:09:09.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
176232025-12-16T03:09:09.351ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
176242025-12-16T03:09:09.351ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
176252025-12-16T03:09:09.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
176262025-12-16T03:09:09.352ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
176272025-12-16T03:09:09.352ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
176282025-12-16T03:09:09.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
176292025-12-16T03:09:09.360ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
176302025-12-16T03:09:09.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
176312025-12-16T03:09:09.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
176322025-12-16T03:09:09.365ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
176332025-12-16T03:09:09.365ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 45.695328ms iteration = 21 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
176342025-12-16T03:09:09.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
176352025-12-16T03:09:09.365ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176362025-12-16T03:09:09.366ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176372025-12-16T03:09:09.369ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
176382025-12-16T03:09:09.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
176392025-12-16T03:09:09.370ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
176402025-12-16T03:09:09.370ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
176412025-12-16T03:09:09.370ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176422025-12-16T03:09:09.370ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
176432025-12-16T03:09:09.371ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
176442025-12-16T03:09:09.371ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
176452025-12-16T03:09:09.371ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-12-16 02:59:18.154440 UTC dns_group = external generation = 2
176462025-12-16T03:09:09.371ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 195.177873ms iteration = 11 status = {"generation":2}
176472025-12-16T03:09:09.372ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176482025-12-16T03:09:09.372ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
176492025-12-16T03:09:09.372ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
176502025-12-16T03:09:09.373ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 154.813251ms iteration = 11 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
176512025-12-16T03:09:09.373ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176522025-12-16T03:09:09.373ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
176532025-12-16T03:09:09.373ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 54.455161ms iteration = 31 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
176542025-12-16T03:09:09.374ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
176552025-12-16T03:09:09.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
176562025-12-16T03:09:09.374ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 55.346381ms iteration = 21 status = 0
176572025-12-16T03:09:09.374ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
176582025-12-16T03:09:09.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
176592025-12-16T03:09:09.375ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
176602025-12-16T03:09:09.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
176612025-12-16T03:09:09.376ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
176622025-12-16T03:09:09.376ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 57.366327ms iteration = 41 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
176632025-12-16T03:09:09.377ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
176642025-12-16T03:09:09.377ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
176652025-12-16T03:09:09.378ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
176662025-12-16T03:09:09.378ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1556, tv_nsec: 987660303 } })
176672025-12-16T03:09:09.378ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
176682025-12-16T03:09:09.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
176692025-12-16T03:09:09.379ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
176702025-12-16T03:09:09.379ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
176712025-12-16T03:09:09.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
176722025-12-16T03:09:09.379ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
176732025-12-16T03:09:09.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
176742025-12-16T03:09:09.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
176752025-12-16T03:09:09.382ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
176762025-12-16T03:09:09.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
176772025-12-16T03:09:09.382ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-12-16T02:59:05.020237Z, time_modified: 2025-12-16T02:59:05.020237Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }) }
176782025-12-16T03:09:09.385ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176792025-12-16T03:09:09.385ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176802025-12-16T03:09:09.390ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 214.003462ms iteration = 11 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]}},"default_endpoint":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
176812025-12-16T03:09:09.393ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
176822025-12-16T03:09:09.393ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
176832025-12-16T03:09:09.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
176842025-12-16T03:09:09.395ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
176852025-12-16T03:09:09.395ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
176862025-12-16T03:09:09.395ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 76.021233ms iteration = 33 status = {"physical_disks_added":0}
176872025-12-16T03:09:09.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
176882025-12-16T03:09:09.396ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
176892025-12-16T03:09:09.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
176902025-12-16T03:09:09.398ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
176912025-12-16T03:09:09.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
176922025-12-16T03:09:09.401ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
176932025-12-16T03:09:09.401ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:473
176942025-12-16T03:09:09.401ZINFOnexus (SagaQuiesceHandle): saga recovery pass finished blueprint_id = Some(a91734db-9ac7-4793-9c32-ddbb21f8d986 (blueprint)) file = nexus/types/src/quiesce.rs:525 generation = 1
176952025-12-16T03:09:09.401ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 166.56727ms iteration = 2 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
176962025-12-16T03:09:09.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
176972025-12-16T03:09:09.404ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176982025-12-16T03:09:09.408ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176992025-12-16T03:09:09.415ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
177002025-12-16T03:09:09.415ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177012025-12-16T03:09:09.415ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
177022025-12-16T03:09:09.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
177032025-12-16T03:09:09.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177042025-12-16T03:09:09.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
177052025-12-16T03:09:09.421ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
177062025-12-16T03:09:09.421ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
177072025-12-16T03:09:09.421ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
177082025-12-16T03:09:09.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
177092025-12-16T03:09:09.427ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
177102025-12-16T03:09:09.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
177112025-12-16T03:09:09.435ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
177122025-12-16T03:09:09.435ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 115.883348ms iteration = 61 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
177132025-12-16T03:09:09.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177142025-12-16T03:09:09.443ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
177152025-12-16T03:09:09.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
177162025-12-16T03:09:09.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
177172025-12-16T03:09:09.445ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177182025-12-16T03:09:09.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177192025-12-16T03:09:09.452ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(377aeb05-9159-41c9-8113-9e905199b37b, 0), (40cdf0c1-4d12-4355-9a49-17ccd3837b4c, 0), (57b1673a-2ae9-4962-b9dd-6b07181145fc, 0), (a8a48239-3eed-426a-8ff8-7f8d17a22586, 0)] collection_id = 377aeb05-9159-41c9-8113-9e905199b37b
177202025-12-16T03:09:09.452ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
177212025-12-16T03:09:09.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
177222025-12-16T03:09:09.454ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 135.099211ms iteration = 21 status = {"errors":[],"finish_invoked_ok":[]}
177232025-12-16T03:09:09.455ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177242025-12-16T03:09:09.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177252025-12-16T03:09:09.460ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177262025-12-16T03:09:09.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177272025-12-16T03:09:09.465ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 00095469-1d05-4e68-8de6-a3957b71e34b step component = SledAgent step id = Ensure
177282025-12-16T03:09:09.466ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177292025-12-16T03:09:09.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177302025-12-16T03:09:09.470ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
177312025-12-16T03:09:09.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
177322025-12-16T03:09:09.471ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177332025-12-16T03:09:09.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177342025-12-16T03:09:09.472ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
177352025-12-16T03:09:09.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
177362025-12-16T03:09:09.474ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
177372025-12-16T03:09:09.474ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177382025-12-16T03:09:09.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
177392025-12-16T03:09:09.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177402025-12-16T03:09:09.476ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 156.893897ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
177412025-12-16T03:09:09.476ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 158.222885ms iteration = 21 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
177422025-12-16T03:09:09.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
177432025-12-16T03:09:09.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
177442025-12-16T03:09:09.478ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177452025-12-16T03:09:09.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177462025-12-16T03:09:09.480ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177472025-12-16T03:09:09.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
177482025-12-16T03:09:09.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
177492025-12-16T03:09:09.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177502025-12-16T03:09:09.487ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
177512025-12-16T03:09:09.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
177522025-12-16T03:09:09.489ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 170.626582ms iteration = 61 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
177532025-12-16T03:09:09.490ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
177542025-12-16T03:09:09.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
177552025-12-16T03:09:09.491ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
177562025-12-16T03:09:09.491ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177572025-12-16T03:09:09.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
177582025-12-16T03:09:09.491ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177592025-12-16T03:09:09.491ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 172.485855ms iteration = 21 status = {"errors":[],"requests_created_ok":[]}
177602025-12-16T03:09:09.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177612025-12-16T03:09:09.493ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177622025-12-16T03:09:09.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177632025-12-16T03:09:09.495ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 175.845107ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
177642025-12-16T03:09:09.497ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177652025-12-16T03:09:09.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177662025-12-16T03:09:09.498ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
177672025-12-16T03:09:09.498ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
177682025-12-16T03:09:09.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
177692025-12-16T03:09:09.499ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177702025-12-16T03:09:09.502ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
177712025-12-16T03:09:09.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
177722025-12-16T03:09:09.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
177732025-12-16T03:09:09.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
177742025-12-16T03:09:09.507ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
177752025-12-16T03:09:09.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
177762025-12-16T03:09:09.509ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
177772025-12-16T03:09:09.509ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177782025-12-16T03:09:09.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177792025-12-16T03:09:09.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177802025-12-16T03:09:09.515ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
177812025-12-16T03:09:09.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
177822025-12-16T03:09:09.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
177832025-12-16T03:09:09.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177842025-12-16T03:09:09.519ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }], Name("external-dns"): [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
177852025-12-16T03:09:09.519ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177862025-12-16T03:09:09.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177872025-12-16T03:09:09.521ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177882025-12-16T03:09:09.524ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
177892025-12-16T03:09:09.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
177902025-12-16T03:09:09.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
177912025-12-16T03:09:09.524ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
177922025-12-16T03:09:09.524ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
177932025-12-16T03:09:09.524ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
177942025-12-16T03:09:09.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
177952025-12-16T03:09:09.525ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
177962025-12-16T03:09:09.525ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177972025-12-16T03:09:09.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177982025-12-16T03:09:09.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
177992025-12-16T03:09:09.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178002025-12-16T03:09:09.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178012025-12-16T03:09:09.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178022025-12-16T03:09:09.530ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
178032025-12-16T03:09:09.531ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
178042025-12-16T03:09:09.531ZDEBGnexus (service-firewall-rule-progator): resolved 3 rules for sleds background_task = service_firewall_rule_propagation
178052025-12-16T03:09:09.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178062025-12-16T03:09:09.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
178072025-12-16T03:09:09.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178082025-12-16T03:09:09.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178092025-12-16T03:09:09.546ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178102025-12-16T03:09:09.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178112025-12-16T03:09:09.549ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
178122025-12-16T03:09:09.549ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178132025-12-16T03:09:09.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
178142025-12-16T03:09:09.556ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
178152025-12-16T03:09:09.559ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
178162025-12-16T03:09:09.593ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
178172025-12-16T03:09:09.593ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
178182025-12-16T03:09:09.593ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 274.7381ms iteration = 21 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
178192025-12-16T03:09:09.604ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
178202025-12-16T03:09:09.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
178212025-12-16T03:09:09.609ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178222025-12-16T03:09:09.609ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
178232025-12-16T03:09:09.609ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
178242025-12-16T03:09:09.610ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "015d038b-5fee-400b-8ca6-c9d1802dd4c7", "content-length": "372", "date": "Tue, 16 Dec 2025 03:09:09 GMT"} })
178252025-12-16T03:09:09.610ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
178262025-12-16T03:09:09.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
178272025-12-16T03:09:09.612ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
178282025-12-16T03:09:09.612ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
178292025-12-16T03:09:09.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
178302025-12-16T03:09:09.615ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178312025-12-16T03:09:09.618ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
178322025-12-16T03:09:09.618ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
178332025-12-16T03:09:09.618ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "98ce5999-7338-45be-9eb8-b1c0fc6ab1b3", "content-length": "26", "date": "Tue, 16 Dec 2025 03:09:09 GMT"} })
178342025-12-16T03:09:09.618ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
178352025-12-16T03:09:09.619ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
178362025-12-16T03:09:09.619ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
178372025-12-16T03:09:09.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
178382025-12-16T03:09:09.621ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
178392025-12-16T03:09:09.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178402025-12-16T03:09:09.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178412025-12-16T03:09:09.631ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
178422025-12-16T03:09:09.631ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
178432025-12-16T03:09:09.632ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c93c0ac2-1cec-4fe1-9374-3aad0e9b3f96", "date": "Tue, 16 Dec 2025 03:09:09 GMT"} })
178442025-12-16T03:09:09.632ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
178452025-12-16T03:09:09.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
178462025-12-16T03:09:09.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178472025-12-16T03:09:09.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178482025-12-16T03:09:09.635ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
178492025-12-16T03:09:09.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
178502025-12-16T03:09:09.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178512025-12-16T03:09:09.639ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
178522025-12-16T03:09:09.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
178532025-12-16T03:09:09.642ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
178542025-12-16T03:09:09.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178552025-12-16T03:09:09.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178562025-12-16T03:09:09.644ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
178572025-12-16T03:09:09.644ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
178582025-12-16T03:09:09.645ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a3e7c1dd-f6fd-4665-ac7c-0944b4d1d5ac", "date": "Tue, 16 Dec 2025 03:09:09 GMT"} })
178592025-12-16T03:09:09.645ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178602025-12-16T03:09:09.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178612025-12-16T03:09:09.647ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 327.749062ms iteration = 31 status = {}
178622025-12-16T03:09:09.647ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178632025-12-16T03:09:09.652ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
178642025-12-16T03:09:09.652ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178652025-12-16T03:09:09.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178662025-12-16T03:09:09.655ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178672025-12-16T03:09:09.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178682025-12-16T03:09:09.657ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178692025-12-16T03:09:09.658ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), time_created: 2025-12-16T02:59:15.195809Z, time_modified: 2025-12-16T02:59:15.195809Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
178702025-12-16T03:09:09.658ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
178712025-12-16T03:09:09.661ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
178722025-12-16T03:09:09.661ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
178732025-12-16T03:09:09.661ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178742025-12-16T03:09:09.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178752025-12-16T03:09:09.664ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178762025-12-16T03:09:09.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178772025-12-16T03:09:09.667ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178782025-12-16T03:09:09.670ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
178792025-12-16T03:09:09.671ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178802025-12-16T03:09:09.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178812025-12-16T03:09:09.706ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
178822025-12-16T03:09:09.707ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "e9157fa8-360d-4ca5-b9e1-07b97f636695", "date": "Tue, 16 Dec 2025 03:09:09 GMT"} })
178832025-12-16T03:09:09.707ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
178842025-12-16T03:09:09.711ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
178852025-12-16T03:09:09.711ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 32 reason = Dependency
178862025-12-16T03:09:09.711ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
178872025-12-16T03:09:09.711ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 472.638028ms iteration = 11 status = {"errors":[],"probes_by_sled":{"b1c76b61-acee-4be6-b582-973c938d6b6e":0}}
178882025-12-16T03:09:09.711ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
178892025-12-16T03:09:09.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
178902025-12-16T03:09:09.713ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178912025-12-16T03:09:09.717ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
178922025-12-16T03:09:09.717ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
178932025-12-16T03:09:09.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
178942025-12-16T03:09:09.720ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178952025-12-16T03:09:09.724ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
178962025-12-16T03:09:09.724ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
178972025-12-16T03:09:09.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
178982025-12-16T03:09:09.725ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 377aeb05-9159-41c9-8113-9e905199b37b file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
178992025-12-16T03:09:09.725ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
179002025-12-16T03:09:09.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
179012025-12-16T03:09:09.770ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
179022025-12-16T03:09:09.770ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
179032025-12-16T03:09:09.770ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 64141384-6a6e-4e15-a47a-2999843b5614 parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
179042025-12-16T03:09:09.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
179052025-12-16T03:09:09.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
179062025-12-16T03:09:09.773ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179072025-12-16T03:09:09.774ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 598.107758ms iteration = 21 status = {}
179082025-12-16T03:09:09.777ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
179092025-12-16T03:09:09.777ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
179102025-12-16T03:09:09.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
179112025-12-16T03:09:09.780ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
179122025-12-16T03:09:09.780ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
179132025-12-16T03:09:09.780ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 563.259486ms iteration = 25 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
179142025-12-16T03:09:09.782ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
179152025-12-16T03:09:09.784ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "acb84e22-0592-4636-b0ce-44db0631fdf2", "date": "Tue, 16 Dec 2025 03:09:09 GMT"} }) sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
179162025-12-16T03:09:09.787ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 00095469-1d05-4e68-8de6-a3957b71e34b step component = FirewallRules step id = Ensure
179172025-12-16T03:09:09.787ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179182025-12-16T03:09:09.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179192025-12-16T03:09:09.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179202025-12-16T03:09:09.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179212025-12-16T03:09:09.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179222025-12-16T03:09:09.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179232025-12-16T03:09:09.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179242025-12-16T03:09:09.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179252025-12-16T03:09:09.797ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179262025-12-16T03:09:09.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179272025-12-16T03:09:09.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179282025-12-16T03:09:09.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179292025-12-16T03:09:09.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179302025-12-16T03:09:09.807ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
179312025-12-16T03:09:09.807ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179322025-12-16T03:09:09.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179332025-12-16T03:09:09.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179342025-12-16T03:09:09.812ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
179352025-12-16T03:09:09.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179362025-12-16T03:09:09.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
179372025-12-16T03:09:09.814ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179382025-12-16T03:09:09.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179392025-12-16T03:09:09.816ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179402025-12-16T03:09:09.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179412025-12-16T03:09:09.818ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
179422025-12-16T03:09:09.819ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
179432025-12-16T03:09:09.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
179442025-12-16T03:09:09.821ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
179452025-12-16T03:09:09.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
179462025-12-16T03:09:09.823ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179472025-12-16T03:09:09.824ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
179482025-12-16T03:09:09.824ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179492025-12-16T03:09:09.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179502025-12-16T03:09:09.826ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
179512025-12-16T03:09:09.826ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
179522025-12-16T03:09:09.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
179532025-12-16T03:09:09.827ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
179542025-12-16T03:09:09.827ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 608.827504ms iteration = 11 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
179552025-12-16T03:09:09.827ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179562025-12-16T03:09:09.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179572025-12-16T03:09:09.828ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
179582025-12-16T03:09:09.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
179592025-12-16T03:09:09.829ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179602025-12-16T03:09:09.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179612025-12-16T03:09:09.831ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179622025-12-16T03:09:09.831ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179632025-12-16T03:09:09.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179642025-12-16T03:09:09.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179652025-12-16T03:09:09.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179662025-12-16T03:09:09.834ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
179672025-12-16T03:09:09.834ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
179682025-12-16T03:09:09.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
179692025-12-16T03:09:09.836ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179702025-12-16T03:09:09.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179712025-12-16T03:09:09.839ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/artifacts-config
179722025-12-16T03:09:09.840ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts-config", status: 204, headers: {"x-request-id": "b3e9231d-c355-4d40-9b82-6bc2dbc33fbd", "date": "Tue, 16 Dec 2025 03:09:09 GMT"} })
179732025-12-16T03:09:09.840ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
179742025-12-16T03:09:09.840ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c5cb9d9-f872-46c9-a5b4-6d526bd47ddc", "content-length": "26", "date": "Tue, 16 Dec 2025 03:09:09 GMT"} })
179752025-12-16T03:09:09.840ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:679 sled = http://[fd00:1122:3344:101::1]:12345
179762025-12-16T03:09:09.840ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179772025-12-16T03:09:09.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179782025-12-16T03:09:09.843ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179792025-12-16T03:09:09.843ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 606.473844ms iteration = 3 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":2,"put_config_err":0,"put_config_ok":2,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"b1c76b61-acee-4be6-b582-973c938d6b6e","time":"2025-12-16T03:04:09.668750966Z"},{"operation":"list","target_sled":"b1c76b61-acee-4be6-b582-973c938d6b6e","time":"2025-12-16T03:04:09.669002590Z"},{"generation":1,"operation":"put_config","target_sled":"b1c76b61-acee-4be6-b582-973c938d6b6e","time":"2025-12-16T03:09:09.840364971Z"},{"operation":"list","target_sled":"b1c76b61-acee-4be6-b582-973c938d6b6e","time":"2025-12-16T03:09:09.840677965Z"}]}
179802025-12-16T03:09:09.843ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(57b1673a-2ae9-4962-b9dd-6b07181145fc, 0), (a8a48239-3eed-426a-8ff8-7f8d17a22586, 0), (1845c551-ac2c-40d7-ad4c-fd64ea891f40, 0)]
179812025-12-16T03:09:09.843ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1557, tv_nsec: 452828027 } })
179822025-12-16T03:09:09.850ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
179832025-12-16T03:09:09.850ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179842025-12-16T03:09:09.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179852025-12-16T03:09:09.852ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179862025-12-16T03:09:09.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179872025-12-16T03:09:09.855ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179882025-12-16T03:09:09.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179892025-12-16T03:09:09.857ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179902025-12-16T03:09:09.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179912025-12-16T03:09:09.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179922025-12-16T03:09:09.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179932025-12-16T03:09:09.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179942025-12-16T03:09:09.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179952025-12-16T03:09:09.863ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179962025-12-16T03:09:09.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179972025-12-16T03:09:09.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179982025-12-16T03:09:09.867ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
179992025-12-16T03:09:09.868ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "aea8e50f-87de-49a8-a006-ef7ee11a87ea", "content-length": "691", "date": "Tue, 16 Dec 2025 03:09:09 GMT"} })
180002025-12-16T03:09:09.868ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
180012025-12-16T03:09:09.868ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
180022025-12-16T03:09:09.868ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
180032025-12-16T03:09:09.869ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
180042025-12-16T03:09:09.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180052025-12-16T03:09:09.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180062025-12-16T03:09:09.870ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 550.855759ms iteration = 21 status = {}
180072025-12-16T03:09:09.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180082025-12-16T03:09:09.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180092025-12-16T03:09:09.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180102025-12-16T03:09:09.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180112025-12-16T03:09:09.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180122025-12-16T03:09:09.876ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
180132025-12-16T03:09:09.876ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "ed5d8934-4b44-4de1-b676-27639228671f", "date": "Tue, 16 Dec 2025 03:09:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
180142025-12-16T03:09:09.876ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
180152025-12-16T03:09:09.879ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "4a534f1f-d643-4259-89ae-a867d8ae7acb", "date": "Tue, 16 Dec 2025 03:09:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
180162025-12-16T03:09:09.879ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
180172025-12-16T03:09:09.881ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
180182025-12-16T03:09:09.881ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 705.203225ms iteration = 13 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
180192025-12-16T03:09:09.881ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
180202025-12-16T03:09:09.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180212025-12-16T03:09:09.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180222025-12-16T03:09:09.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180232025-12-16T03:09:09.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180242025-12-16T03:09:09.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180252025-12-16T03:09:09.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180262025-12-16T03:09:09.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180272025-12-16T03:09:09.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180282025-12-16T03:09:09.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180292025-12-16T03:09:09.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180302025-12-16T03:09:09.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180312025-12-16T03:09:09.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180322025-12-16T03:09:09.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180332025-12-16T03:09:09.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180342025-12-16T03:09:09.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180352025-12-16T03:09:09.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180362025-12-16T03:09:09.900ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180372025-12-16T03:09:09.905ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
180382025-12-16T03:09:09.905ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180392025-12-16T03:09:09.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180402025-12-16T03:09:09.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180412025-12-16T03:09:09.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180422025-12-16T03:09:09.908ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180432025-12-16T03:09:09.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180442025-12-16T03:09:09.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180452025-12-16T03:09:09.914ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
180462025-12-16T03:09:09.914ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180472025-12-16T03:09:09.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180482025-12-16T03:09:09.921ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], Name("nexus"): [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
180492025-12-16T03:09:09.921ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180502025-12-16T03:09:09.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180512025-12-16T03:09:09.923ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180522025-12-16T03:09:09.925ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
180532025-12-16T03:09:09.925ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180542025-12-16T03:09:09.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180552025-12-16T03:09:09.926ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
180562025-12-16T03:09:09.926ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
180572025-12-16T03:09:09.926ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
180582025-12-16T03:09:09.928ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
180592025-12-16T03:09:09.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
180602025-12-16T03:09:09.931ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
180612025-12-16T03:09:09.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
180622025-12-16T03:09:09.936ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180632025-12-16T03:09:09.943ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
180642025-12-16T03:09:09.943ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
180652025-12-16T03:09:09.945ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "99ad0707-8b77-4fc1-9f5d-dffa950415b9", "date": "Tue, 16 Dec 2025 03:09:09 GMT"} })
180662025-12-16T03:09:09.946ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:506
180672025-12-16T03:09:09.946ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 728.653402ms
180682025-12-16T03:09:09.946ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 728.666012ms iteration = 3 status = {}
180692025-12-16T03:09:09.952ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), time_created: 2025-12-16T02:59:15.195809Z, time_modified: 2025-12-16T02:59:15.195809Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
180702025-12-16T03:09:09.952ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
180712025-12-16T03:09:10.000ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
180722025-12-16T03:09:10.000ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
180732025-12-16T03:09:10.007ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c3f98fb-7cea-455b-951e-12904ac86ba6", "content-length": "372", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
180742025-12-16T03:09:10.007ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
180752025-12-16T03:09:10.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
180762025-12-16T03:09:10.021ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
180772025-12-16T03:09:10.022ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1557, tv_nsec: 631072706 } })
180782025-12-16T03:09:10.083ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
180792025-12-16T03:09:10.083ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
180802025-12-16T03:09:10.084ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "ed5334e0-f89e-482b-99f8-68a025abc9b4", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
180812025-12-16T03:09:10.084ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
180822025-12-16T03:09:10.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
180832025-12-16T03:09:10.087ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
180842025-12-16T03:09:10.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
180852025-12-16T03:09:10.091ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
180862025-12-16T03:09:10.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
180872025-12-16T03:09:10.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180882025-12-16T03:09:10.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180892025-12-16T03:09:10.094ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
180902025-12-16T03:09:10.094ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
180912025-12-16T03:09:10.096ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
180922025-12-16T03:09:10.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
180932025-12-16T03:09:10.096ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "68ae0842-11ec-42e5-8b50-6daed8ae001e", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
180942025-12-16T03:09:10.098ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180952025-12-16T03:09:10.099ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 388.154508ms iteration = 32 status = {}
180962025-12-16T03:09:10.101ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
180972025-12-16T03:09:10.101ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
180982025-12-16T03:09:10.101ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("20488683-5778-4090-b36e-e6f0d80a682b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("35dbaad1-8536-44a0-8a0a-35d7c39dcfbb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b57fa226-2428-4a54-b85d-60409e28b484.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("d812a3e2-48ae-4812-a1cc-7eb7623e4e00.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::5)) }], valid_until: Instant { tv_sec: 1557, tv_nsec: 710658294 } })
180992025-12-16T03:09:10.102ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
181002025-12-16T03:09:10.102ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
181012025-12-16T03:09:10.102ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
181022025-12-16T03:09:10.102ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "59124f7e-d97b-4500-a804-efd836b5c21b", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181032025-12-16T03:09:10.102ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d18757c-ca79-4e8f-99e7-0b625c711b14", "content-length": "476", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181042025-12-16T03:09:10.102ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
181052025-12-16T03:09:10.103ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4394ca66-081e-4782-988c-eb990a46f59e", "content-length": "734", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181062025-12-16T03:09:10.103ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
181072025-12-16T03:09:10.103ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc2d4ae0-c71c-4385-91bc-6d51a33fdb4f", "content-length": "111", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181082025-12-16T03:09:10.103ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
181092025-12-16T03:09:10.103ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
181102025-12-16T03:09:10.103ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 00095469-1d05-4e68-8de6-a3957b71e34b step component = Dns step id = Ensure
181112025-12-16T03:09:10.103ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
181122025-12-16T03:09:10.103ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181132025-12-16T03:09:10.104ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "73bfdab0-7b9a-4639-b3c2-f9677238e8da", "content-length": "111", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181142025-12-16T03:09:10.104ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
181152025-12-16T03:09:10.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181162025-12-16T03:09:10.104ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee725cab-9465-4292-8345-a0a350ae57dd", "content-length": "163", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181172025-12-16T03:09:10.104ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
181182025-12-16T03:09:10.104ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181192025-12-16T03:09:10.104ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c3ea027-af65-4bf5-9430-4cecdda0fa22", "content-length": "163", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181202025-12-16T03:09:10.104ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
181212025-12-16T03:09:10.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181222025-12-16T03:09:10.105ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f755273e-20b3-4cb8-8b20-5662b5a199f4", "content-length": "166", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181232025-12-16T03:09:10.105ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
181242025-12-16T03:09:10.105ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "944c7f11-3018-4d76-a854-602f10aef4e4", "content-length": "166", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181252025-12-16T03:09:10.105ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
181262025-12-16T03:09:10.105ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e45394e-ee0d-473d-aae8-14b1589ecdfa", "content-length": "702", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181272025-12-16T03:09:10.105ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
181282025-12-16T03:09:10.106ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8ce288f-84df-481e-883f-13920f814f98", "content-length": "727", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181292025-12-16T03:09:10.106ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
181302025-12-16T03:09:10.106ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f98b0627-df78-45f1-9c57-743554d0d609", "content-length": "729", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181312025-12-16T03:09:10.106ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
181322025-12-16T03:09:10.106ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b03846e1-6dd4-41ac-b2fb-12e4e4eced9e", "content-length": "728", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181332025-12-16T03:09:10.106ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
181342025-12-16T03:09:10.106ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
181352025-12-16T03:09:10.106ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "411ca6dd-6273-4a4e-bb01-98455b38ea05", "content-length": "722", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181362025-12-16T03:09:10.106ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
181372025-12-16T03:09:10.107ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ef1c07c-480e-4f58-bbe1-f1d8f406c959", "content-length": "10", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181382025-12-16T03:09:10.107ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
181392025-12-16T03:09:10.107ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f8051859-4695-48d9-a351-c4bf9e1c91ce", "content-length": "144", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181402025-12-16T03:09:10.107ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
181412025-12-16T03:09:10.107ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3db2656c-7b55-421e-9a1b-6a0c95ffb719", "content-length": "147", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181422025-12-16T03:09:10.107ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
181432025-12-16T03:09:10.107ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f8b4ac1-d679-4212-817d-96178df1febb", "content-length": "109", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181442025-12-16T03:09:10.107ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
181452025-12-16T03:09:10.108ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9ab84a7-c54c-4daa-ac51-3654f6f06ccf", "content-length": "109", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181462025-12-16T03:09:10.108ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
181472025-12-16T03:09:10.108ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5eed070-fd9e-42bd-95ff-114678a9aa45", "content-length": "163", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181482025-12-16T03:09:10.108ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
181492025-12-16T03:09:10.108ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5703d626-ee6b-41c2-b6ef-09e63d54c615", "content-length": "163", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181502025-12-16T03:09:10.108ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
181512025-12-16T03:09:10.109ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "944a0d36-e365-4959-8098-6f01f73b9766", "content-length": "166", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181522025-12-16T03:09:10.109ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
181532025-12-16T03:09:10.109ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a34f05a-b5ea-4129-a6f2-adee3e472ea9", "content-length": "166", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181542025-12-16T03:09:10.109ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
181552025-12-16T03:09:10.110ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "91d564f1-bf60-41dc-8f19-97863ac6fbc9", "content-length": "702", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181562025-12-16T03:09:10.110ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
181572025-12-16T03:09:10.110ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "37026923-79dd-4e17-8cd8-a5ab57462e17", "content-length": "727", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181582025-12-16T03:09:10.110ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
181592025-12-16T03:09:10.110ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
181602025-12-16T03:09:10.110ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c27b4db3-2aa5-4715-a1c2-8d12e0149ff1", "content-length": "729", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181612025-12-16T03:09:10.110ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
181622025-12-16T03:09:10.110ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a96cb9db-8d79-420d-97d5-eebb89bf286c", "content-length": "728", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
181632025-12-16T03:09:10.110ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
181642025-12-16T03:09:10.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
181652025-12-16T03:09:10.112ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
181662025-12-16T03:09:10.112ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
181672025-12-16T03:09:10.112ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "6fa04982-873f-4840-9aff-82d094e86131", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
181682025-12-16T03:09:10.112ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
181692025-12-16T03:09:10.114ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "1358e0ad-5e08-40de-af34-3fab543a3178", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
181702025-12-16T03:09:10.114ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
181712025-12-16T03:09:10.114ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
181722025-12-16T03:09:10.114ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
181732025-12-16T03:09:10.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
181742025-12-16T03:09:10.116ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
181752025-12-16T03:09:10.116ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "0b76f076-04b4-4527-8d5d-6206a34cb7cd", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
181762025-12-16T03:09:10.116ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
181772025-12-16T03:09:10.118ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
181782025-12-16T03:09:10.118ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 942.482211ms iteration = 13 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
181792025-12-16T03:09:10.118ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
181802025-12-16T03:09:10.118ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
181812025-12-16T03:09:10.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
181822025-12-16T03:09:10.124ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1557, tv_nsec: 732965697 } })
181832025-12-16T03:09:10.158ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
181842025-12-16T03:09:10.158ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
181852025-12-16T03:09:10.158ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
181862025-12-16T03:09:10.158ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181872025-12-16T03:09:10.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181882025-12-16T03:09:10.159ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
181892025-12-16T03:09:10.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
181902025-12-16T03:09:10.160ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
181912025-12-16T03:09:10.161ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
181922025-12-16T03:09:10.161ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
181932025-12-16T03:09:10.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
181942025-12-16T03:09:10.163ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
181952025-12-16T03:09:10.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181962025-12-16T03:09:10.163ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181972025-12-16T03:09:10.179ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
181982025-12-16T03:09:10.179ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
181992025-12-16T03:09:10.180ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
182002025-12-16T03:09:10.180ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
182012025-12-16T03:09:10.180ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
182022025-12-16T03:09:10.180ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
182032025-12-16T03:09:10.180ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
182042025-12-16T03:09:10.180ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
182052025-12-16T03:09:10.181ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
182062025-12-16T03:09:10.181ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
182072025-12-16T03:09:10.181ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
182082025-12-16T03:09:10.181ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
182092025-12-16T03:09:10.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
182102025-12-16T03:09:10.185ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
182112025-12-16T03:09:10.185ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
182122025-12-16T03:09:10.210ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
182132025-12-16T03:09:10.210ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
182142025-12-16T03:09:10.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182152025-12-16T03:09:10.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182162025-12-16T03:09:10.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182172025-12-16T03:09:10.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
182182025-12-16T03:09:10.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182192025-12-16T03:09:10.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182202025-12-16T03:09:10.220ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
182212025-12-16T03:09:10.220ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
182222025-12-16T03:09:10.220ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
182232025-12-16T03:09:10.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182242025-12-16T03:09:10.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182252025-12-16T03:09:10.221ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182262025-12-16T03:09:10.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182272025-12-16T03:09:10.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182282025-12-16T03:09:10.226ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
182292025-12-16T03:09:10.245ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
182302025-12-16T03:09:10.245ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
182312025-12-16T03:09:10.274ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
182322025-12-16T03:09:10.274ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
182332025-12-16T03:09:10.274ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
182342025-12-16T03:09:10.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182352025-12-16T03:09:10.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182362025-12-16T03:09:10.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182372025-12-16T03:09:10.280ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
182382025-12-16T03:09:10.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182392025-12-16T03:09:10.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182402025-12-16T03:09:10.306ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "adbd7497-7304-4eeb-8e35-5c3164766155", "content-length": "95612", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
182412025-12-16T03:09:10.308ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
182422025-12-16T03:09:10.308ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
182432025-12-16T03:09:10.308ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
182442025-12-16T03:09:10.308ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
182452025-12-16T03:09:10.308ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
182462025-12-16T03:09:10.308ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
182472025-12-16T03:09:10.308ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
182482025-12-16T03:09:10.308ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
182492025-12-16T03:09:10.309ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "dcfc4d9c-337a-4c9f-8f06-313f1e9ae596", "content-length": "64", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
182502025-12-16T03:09:10.309ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c249b0e-a7b7-47f3-96a8-16e6e53095be", "content-length": "64", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
182512025-12-16T03:09:10.309ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
182522025-12-16T03:09:10.309ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bdf17c2f-0f79-40ca-be78-b2d999396b5b", "content-length": "64", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
182532025-12-16T03:09:10.309ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
182542025-12-16T03:09:10.309ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab7af933-a351-4bd1-8042-382d2eead32c", "content-length": "64", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
182552025-12-16T03:09:10.309ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e234d199-696b-4978-9ef0-2506f8d72fbb", "content-length": "64", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
182562025-12-16T03:09:10.309ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
182572025-12-16T03:09:10.309ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
182582025-12-16T03:09:10.309ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
182592025-12-16T03:09:10.312ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "7b217c5e-55b4-4e22-ac37-76588973c743", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
182602025-12-16T03:09:10.312ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
182612025-12-16T03:09:10.312ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
182622025-12-16T03:09:10.313ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "94343d89-17ef-4388-88a6-d1a4b99f1759", "content-length": "682", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
182632025-12-16T03:09:10.313ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
182642025-12-16T03:09:10.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
182652025-12-16T03:09:10.317ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
182662025-12-16T03:09:10.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
182672025-12-16T03:09:10.318ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
182682025-12-16T03:09:10.318ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
182692025-12-16T03:09:10.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182702025-12-16T03:09:10.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182712025-12-16T03:09:10.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182722025-12-16T03:09:10.321ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
182732025-12-16T03:09:10.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
182742025-12-16T03:09:10.323ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
182752025-12-16T03:09:10.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
182762025-12-16T03:09:10.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
182772025-12-16T03:09:10.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182782025-12-16T03:09:10.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182792025-12-16T03:09:10.329ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
182802025-12-16T03:09:10.329ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
182812025-12-16T03:09:10.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182822025-12-16T03:09:10.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182832025-12-16T03:09:10.330ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.013705694s iteration = 21 status = {}
182842025-12-16T03:09:10.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182852025-12-16T03:09:10.334ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
182862025-12-16T03:09:10.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182872025-12-16T03:09:10.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182882025-12-16T03:09:10.336ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182892025-12-16T03:09:10.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182902025-12-16T03:09:10.337ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182912025-12-16T03:09:10.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182922025-12-16T03:09:10.338ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182932025-12-16T03:09:10.343ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
182942025-12-16T03:09:10.344ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
182952025-12-16T03:09:10.344ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
182962025-12-16T03:09:10.344ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 00095469-1d05-4e68-8de6-a3957b71e34b step component = OmicronZones step id = Cleanup
182972025-12-16T03:09:10.344ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 00095469-1d05-4e68-8de6-a3957b71e34b step component = OmicronZones step id = Cleanup
182982025-12-16T03:09:10.344ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 00095469-1d05-4e68-8de6-a3957b71e34b step component = PhysicalDisks step id = Cleanup
182992025-12-16T03:09:10.344ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 00095469-1d05-4e68-8de6-a3957b71e34b step component = Clickhouse step id = Ensure
183002025-12-16T03:09:10.344ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 00095469-1d05-4e68-8de6-a3957b71e34b step component = Clickhouse step id = Ensure
183012025-12-16T03:09:10.348ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2d4e088-e671-4ee7-bb77-e2ea81411174", "content-length": "408431", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
183022025-12-16T03:09:10.349ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0d63a99-33b3-40bc-902d-5834d7e879d0", "content-length": "409481", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
183032025-12-16T03:09:10.349ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "863faf70-2d47-47b3-bbe4-4b1d1fb11576", "content-length": "437795", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
183042025-12-16T03:09:10.350ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6ca5c51-875c-4640-ad1d-8ee101296e53", "content-length": "409395", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
183052025-12-16T03:09:10.351ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "25ed244b-fc20-4fe5-af2e-4d9fad505fd8", "content-length": "407350", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
183062025-12-16T03:09:10.353ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
183072025-12-16T03:09:10.353ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
183082025-12-16T03:09:10.353ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
183092025-12-16T03:09:10.353ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
183102025-12-16T03:09:10.353ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
183112025-12-16T03:09:10.393ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
183122025-12-16T03:09:10.400ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "96e258e3-aa85-4129-8284-f253205acfe5", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
183132025-12-16T03:09:10.401ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 00095469-1d05-4e68-8de6-a3957b71e34b step component = SupportBundles step id = Cleanup
183142025-12-16T03:09:10.401ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
183152025-12-16T03:09:10.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
183162025-12-16T03:09:10.404ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183172025-12-16T03:09:10.405ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
183182025-12-16T03:09:10.406ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
183192025-12-16T03:09:10.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
183202025-12-16T03:09:10.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
183212025-12-16T03:09:10.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
183222025-12-16T03:09:10.408ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c (omicron_zone)
183232025-12-16T03:09:10.408ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
183242025-12-16T03:09:10.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183252025-12-16T03:09:10.411ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ba52994-781d-400a-a160-e5aef7a16e98", "content-length": "123", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
183262025-12-16T03:09:10.412ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
183272025-12-16T03:09:10.413ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
183282025-12-16T03:09:10.456ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 00095469-1d05-4e68-8de6-a3957b71e34b step component = OmicronZones step id = Cleanup
183292025-12-16T03:09:10.456ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
183302025-12-16T03:09:10.456ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
183312025-12-16T03:09:10.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
183322025-12-16T03:09:10.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
183332025-12-16T03:09:10.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183342025-12-16T03:09:10.460ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
183352025-12-16T03:09:10.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
183362025-12-16T03:09:10.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
183372025-12-16T03:09:10.505ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
183382025-12-16T03:09:10.505ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(a91734db-9ac7-4793-9c32-ddbb21f8d986 (blueprint), false)
183392025-12-16T03:09:10.505ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 00095469-1d05-4e68-8de6-a3957b71e34b step component = Cockroach step id = Ensure
183402025-12-16T03:09:10.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
183412025-12-16T03:09:10.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
183422025-12-16T03:09:10.518ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
183432025-12-16T03:09:10.518ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
183442025-12-16T03:09:10.519ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "288fe4ca-981a-4136-a1bd-0e54549a7a4b", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
183452025-12-16T03:09:10.519ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
183462025-12-16T03:09:10.521ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
183472025-12-16T03:09:10.521ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
183482025-12-16T03:09:10.521ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "09395025-6a10-470c-803e-cd5b8346f2b9", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
183492025-12-16T03:09:10.521ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
183502025-12-16T03:09:10.524ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
183512025-12-16T03:09:10.524ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
183522025-12-16T03:09:10.524ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "148fd2fd-2a42-41dc-aca6-5336eb37b780", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:09:10 GMT"} })
183532025-12-16T03:09:10.524ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
183542025-12-16T03:09:10.526ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
183552025-12-16T03:09:10.526ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
183562025-12-16T03:09:10.528ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
183572025-12-16T03:09:10.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
183582025-12-16T03:09:10.530ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183592025-12-16T03:09:10.530ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
183602025-12-16T03:09:10.530ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 00095469-1d05-4e68-8de6-a3957b71e34b step component = MgsUpdates step id = Ensure
183612025-12-16T03:09:10.530ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 32 reason = Dependency
183622025-12-16T03:09:10.530ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
183632025-12-16T03:09:10.530ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.354644583s iteration = 12 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"411ab478-9741-403b-baf8-902590c43d30","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"411ab478-9741-403b-baf8-902590c43d30","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":4560,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":64226124,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":64226084,"secs":0}},"event_index":1,"execution_id":"411ab478-9741-403b-baf8-902590c43d30","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":64244014,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":47941397,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":47941348,"secs":0}},"event_index":2,"execution_id":"411ab478-9741-403b-baf8-902590c43d30","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":112243143,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":176840890,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":176840850,"secs":0}},"event_index":3,"execution_id":"411ab478-9741-403b-baf8-902590c43d30","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":289111313,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":321775018,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":321774978,"secs":0}},"event_index":4,"execution_id":"411ab478-9741-403b-baf8-902590c43d30","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":610910490,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":316791073,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":316791033,"secs":0}},"event_index":5,"execution_id":"411ab478-9741-403b-baf8-902590c43d30","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":927716735,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":240834151,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":240834111,"secs":0}},"event_index":6,"execution_id":"411ab478-9741-403b-baf8-902590c43d30","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":168557096,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4240,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4210,"secs":0}},"event_index":7,"execution_id":"411ab478-9741-403b-baf8-902590c43d30","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":168573816,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2101,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2061,"secs":0}},"event_index":8,"execution_id":"411ab478-9741-403b-baf8-902590c43d30","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":168578407,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3880,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3840,"secs":0}},"event_index":9,"execution_id":"411ab478-9741-403b-baf8-902590c43d30","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":168584416,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1660,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1620,"secs":0}},"event_index":10,"execution_id":"411ab478-9741-403b-baf8-902590c43d30","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":168587727,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":57076384,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":57076354,"secs":0}},"event_index":11,"execution_id":"411ab478-9741-403b-baf8-902590c43d30","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":225665800,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":54296017,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":54295977,"secs":0}},"event_index":12,"execution_id":"411ab478-9741-403b-baf8-902590c43d30","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":279974767,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":49597919,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":49597879,"secs":0}},"event_index":13,"execution_id":"411ab478-9741-403b-baf8-902590c43d30","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":329596896,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":24696273,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":24696243,"secs":0}},"event_index":14,"execution_id":"411ab478-9741-403b-baf8-902590c43d30","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":354320009,"secs":1}},{"data":{"attempt_elapsed":{"nanos":8130,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":8090,"secs":0}},"event_index":15,"execution_id":"411ab478-9741-403b-baf8-902590c43d30","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":354346519,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986"}
183642025-12-16T03:09:10.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
183652025-12-16T03:09:10.532ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183662025-12-16T03:09:10.533ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
183672025-12-16T03:09:10.533ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
183682025-12-16T03:09:10.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
183692025-12-16T03:09:10.535ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
183702025-12-16T03:09:10.535ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
183712025-12-16T03:09:10.535ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
183722025-12-16T03:09:10.537ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
183732025-12-16T03:09:10.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
183742025-12-16T03:09:10.539ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183752025-12-16T03:09:10.541ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
183762025-12-16T03:09:10.541ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
183772025-12-16T03:09:10.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
183782025-12-16T03:09:10.543ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
183792025-12-16T03:09:10.543ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 12.465588ms iteration = 32 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
183802025-12-16T03:09:10.753ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 71765244-afb4-4b31-85ba-c7a3812878cc file = nexus/db-queries/src/db/datastore/inventory.rs:1613
183812025-12-16T03:09:10.753ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 71765244-afb4-4b31-85ba-c7a3812878cc time_started = 2025-12-16 03:09:10.102296 UTC
183822025-12-16T03:09:10.753ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 53 reason = Dependency
183832025-12-16T03:09:10.753ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
183842025-12-16T03:09:10.753ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 35 reason = Dependency
183852025-12-16T03:09:10.753ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
183862025-12-16T03:09:10.753ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 15.52\u{b5}s iteration = 35 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
183872025-12-16T03:09:10.753ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.535798609s iteration = 12 status = {"collection_id":"71765244-afb4-4b31-85ba-c7a3812878cc","time_done":"2025-12-16 03:09:10.526529 UTC","time_started":"2025-12-16 03:09:10.102296 UTC"}
183882025-12-16T03:09:10.753ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 13 reason = Dependency
183892025-12-16T03:09:10.753ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
183902025-12-16T03:09:10.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
183912025-12-16T03:09:10.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
183922025-12-16T03:09:10.755ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183932025-12-16T03:09:10.755ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183942025-12-16T03:09:10.758ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
183952025-12-16T03:09:10.758ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
183962025-12-16T03:09:10.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
183972025-12-16T03:09:10.759ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
183982025-12-16T03:09:10.759ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
183992025-12-16T03:09:10.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
184002025-12-16T03:09:10.865ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
184012025-12-16T03:09:10.865ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(a8a48239-3eed-426a-8ff8-7f8d17a22586, 0), (1845c551-ac2c-40d7-ad4c-fd64ea891f40, 0), (90d8f4a5-997b-4b42-8984-c26bd19b75b5, 0), (11a597ae-25b1-4a2f-bcfa-b171e54fe7b9, 0)] collection_id = a8a48239-3eed-426a-8ff8-7f8d17a22586
184022025-12-16T03:09:10.865ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
184032025-12-16T03:09:10.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
184042025-12-16T03:09:10.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
184052025-12-16T03:09:10.917ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 34 reason = Dependency
184062025-12-16T03:09:10.917ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 164.532744ms iteration = 53 status = {"Loaded":{"collection_id":"71765244-afb4-4b31-85ba-c7a3812878cc","time_loaded":"2025-12-16T03:09:10.753166526Z","time_started":"2025-12-16T03:09:10.102296Z"}}
184072025-12-16T03:09:10.917ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 16 reason = Dependency
184082025-12-16T03:09:10.917ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 36 reason = Dependency
184092025-12-16T03:09:10.917ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
184102025-12-16T03:09:10.917ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 7.43\u{b5}s iteration = 36 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
184112025-12-16T03:09:10.917ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
184122025-12-16T03:09:10.917ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 26 reason = Dependency
184132025-12-16T03:09:10.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
184142025-12-16T03:09:10.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
184152025-12-16T03:09:10.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
184162025-12-16T03:09:10.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
184172025-12-16T03:09:10.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
184182025-12-16T03:09:10.919ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184192025-12-16T03:09:10.919ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184202025-12-16T03:09:10.919ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184212025-12-16T03:09:10.923ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
184222025-12-16T03:09:10.923ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
184232025-12-16T03:09:10.923ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
184242025-12-16T03:09:10.923ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
184252025-12-16T03:09:10.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
184262025-12-16T03:09:10.923ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
184272025-12-16T03:09:10.923ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
184282025-12-16T03:09:10.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
184292025-12-16T03:09:10.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
184302025-12-16T03:09:10.925ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184312025-12-16T03:09:10.925ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
184322025-12-16T03:09:10.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
184332025-12-16T03:09:10.927ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184342025-12-16T03:09:10.928ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
184352025-12-16T03:09:10.928ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
184362025-12-16T03:09:10.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
184372025-12-16T03:09:10.929ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
184382025-12-16T03:09:10.929ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
184392025-12-16T03:09:10.930ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
184402025-12-16T03:09:10.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
184412025-12-16T03:09:10.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
184422025-12-16T03:09:10.931ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184432025-12-16T03:09:10.933ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
184442025-12-16T03:09:10.933ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
184452025-12-16T03:09:10.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
184462025-12-16T03:09:10.935ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184472025-12-16T03:09:10.937ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
184482025-12-16T03:09:10.937ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
184492025-12-16T03:09:10.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
184502025-12-16T03:09:10.939ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 21.29948ms iteration = 34 status = {"physical_disks_added":0}
184512025-12-16T03:09:10.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184522025-12-16T03:09:10.967ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
184532025-12-16T03:09:10.967ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
184542025-12-16T03:09:10.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
184552025-12-16T03:09:10.969ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
184562025-12-16T03:09:10.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
184572025-12-16T03:09:10.971ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184582025-12-16T03:09:10.973ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
184592025-12-16T03:09:10.973ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
184602025-12-16T03:09:10.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
184612025-12-16T03:09:10.975ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184622025-12-16T03:09:10.977ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
184632025-12-16T03:09:10.977ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
184642025-12-16T03:09:10.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
184652025-12-16T03:09:10.977ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
184662025-12-16T03:09:10.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
184672025-12-16T03:09:10.980ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
184682025-12-16T03:09:10.980ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
184692025-12-16T03:09:10.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
184702025-12-16T03:09:10.980ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
184712025-12-16T03:09:10.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
184722025-12-16T03:09:10.981ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184732025-12-16T03:09:10.982ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184742025-12-16T03:09:10.984ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
184752025-12-16T03:09:10.984ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
184762025-12-16T03:09:10.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
184772025-12-16T03:09:10.985ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
184782025-12-16T03:09:10.985ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
184792025-12-16T03:09:10.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
184802025-12-16T03:09:10.986ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184812025-12-16T03:09:10.987ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
184822025-12-16T03:09:10.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
184832025-12-16T03:09:10.988ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
184842025-12-16T03:09:10.988ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
184852025-12-16T03:09:10.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
184862025-12-16T03:09:10.989ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184872025-12-16T03:09:10.989ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
184882025-12-16T03:09:10.989ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 72.178344ms iteration = 16 status = {"blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","inventory_collection_id":"71765244-afb4-4b31-85ba-c7a3812878cc","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
184892025-12-16T03:09:10.992ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
184902025-12-16T03:09:10.992ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
184912025-12-16T03:09:10.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
184922025-12-16T03:09:10.994ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184932025-12-16T03:09:10.999ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
184942025-12-16T03:09:10.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
184952025-12-16T03:09:10.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
184962025-12-16T03:09:11.001ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
184972025-12-16T03:09:11.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
184982025-12-16T03:09:11.002ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184992025-12-16T03:09:11.006ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
185002025-12-16T03:09:11.006ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185012025-12-16T03:09:11.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185022025-12-16T03:09:11.008ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185032025-12-16T03:09:11.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185042025-12-16T03:09:11.010ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185052025-12-16T03:09:11.013ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
185062025-12-16T03:09:11.013ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185072025-12-16T03:09:11.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185082025-12-16T03:09:11.014ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185092025-12-16T03:09:11.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185102025-12-16T03:09:11.016ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185112025-12-16T03:09:11.021ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
185122025-12-16T03:09:11.021ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185132025-12-16T03:09:11.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185142025-12-16T03:09:11.038ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185152025-12-16T03:09:11.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185162025-12-16T03:09:11.041ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185172025-12-16T03:09:11.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185182025-12-16T03:09:11.043ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185192025-12-16T03:09:11.046ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
185202025-12-16T03:09:11.046ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185212025-12-16T03:09:11.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185222025-12-16T03:09:11.049ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185232025-12-16T03:09:11.052ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
185242025-12-16T03:09:11.052ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185252025-12-16T03:09:11.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185262025-12-16T03:09:11.054ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185272025-12-16T03:09:11.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185282025-12-16T03:09:11.056ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185292025-12-16T03:09:11.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185302025-12-16T03:09:11.057ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185312025-12-16T03:09:11.061ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
185322025-12-16T03:09:11.061ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185332025-12-16T03:09:11.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185342025-12-16T03:09:11.062ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185352025-12-16T03:09:11.066ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
185362025-12-16T03:09:11.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185372025-12-16T03:09:11.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185382025-12-16T03:09:11.068ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 389 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 7f29778c-a49b-40ee-b34e-3a5420632099 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
185392025-12-16T03:09:11.069ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185402025-12-16T03:09:11.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185412025-12-16T03:09:11.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185422025-12-16T03:09:11.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185432025-12-16T03:09:11.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185442025-12-16T03:09:11.076ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
185452025-12-16T03:09:11.088ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = a8a48239-3eed-426a-8ff8-7f8d17a22586 file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
185462025-12-16T03:09:11.088ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
185472025-12-16T03:09:11.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
185482025-12-16T03:09:11.116ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185492025-12-16T03:09:11.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185502025-12-16T03:09:11.117ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185512025-12-16T03:09:11.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185522025-12-16T03:09:11.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185532025-12-16T03:09:11.122ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
185542025-12-16T03:09:11.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185552025-12-16T03:09:11.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185562025-12-16T03:09:11.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185572025-12-16T03:09:11.126ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185582025-12-16T03:09:11.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185592025-12-16T03:09:11.130ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
185602025-12-16T03:09:11.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185612025-12-16T03:09:11.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185622025-12-16T03:09:11.132ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185632025-12-16T03:09:11.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185642025-12-16T03:09:11.134ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185652025-12-16T03:09:11.137ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
185662025-12-16T03:09:11.137ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185672025-12-16T03:09:11.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185682025-12-16T03:09:11.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185692025-12-16T03:09:11.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185702025-12-16T03:09:11.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185712025-12-16T03:09:11.145ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
185722025-12-16T03:09:11.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185732025-12-16T03:09:11.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185742025-12-16T03:09:11.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185752025-12-16T03:09:11.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185762025-12-16T03:09:11.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185772025-12-16T03:09:11.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
185782025-12-16T03:09:11.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185792025-12-16T03:09:11.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185802025-12-16T03:09:11.179ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(90d8f4a5-997b-4b42-8984-c26bd19b75b5, 0), (11a597ae-25b1-4a2f-bcfa-b171e54fe7b9, 0), (71765244-afb4-4b31-85ba-c7a3812878cc, 0)]
185812025-12-16T03:09:11.180ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1558, tv_nsec: 789060477 } })
185822025-12-16T03:09:11.193ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
185832025-12-16T03:09:11.194ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
185842025-12-16T03:09:11.194ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 4c55fa4a-d378-4666-b03e-2fc3a402081f parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
185852025-12-16T03:09:11.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185862025-12-16T03:09:11.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185872025-12-16T03:09:11.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185882025-12-16T03:09:11.199ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
185892025-12-16T03:09:11.199ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
185902025-12-16T03:09:11.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
185912025-12-16T03:09:11.202ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
185922025-12-16T03:09:11.202ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
185932025-12-16T03:09:11.202ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 284.579055ms iteration = 26 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
185942025-12-16T03:09:11.212ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("20488683-5778-4090-b36e-e6f0d80a682b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("35dbaad1-8536-44a0-8a0a-35d7c39dcfbb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b57fa226-2428-4a54-b85d-60409e28b484.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("d812a3e2-48ae-4812-a1cc-7eb7623e4e00.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::5)) }], valid_until: Instant { tv_sec: 1558, tv_nsec: 821864943 } })
185952025-12-16T03:09:11.213ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
185962025-12-16T03:09:11.213ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
185972025-12-16T03:09:11.213ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
185982025-12-16T03:09:11.213ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "84847732-697b-45f8-99be-d89a2a6b924b", "content-length": "476", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
185992025-12-16T03:09:11.213ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
186002025-12-16T03:09:11.214ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a0510be-529b-4ace-ba75-574cd8312cf3", "content-length": "734", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186012025-12-16T03:09:11.214ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
186022025-12-16T03:09:11.214ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4b5f87f-fe89-4afe-8573-5a2e2b0fb9f8", "content-length": "111", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186032025-12-16T03:09:11.214ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
186042025-12-16T03:09:11.214ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c935e09e-d858-43e3-8620-5edcf6ae191b", "content-length": "111", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186052025-12-16T03:09:11.214ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
186062025-12-16T03:09:11.215ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "01015884-e653-4bf2-a756-f1e9cb9ed285", "content-length": "163", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186072025-12-16T03:09:11.215ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
186082025-12-16T03:09:11.215ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f1811d1-74b1-4850-a40a-d4b05715177d", "content-length": "163", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186092025-12-16T03:09:11.215ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
186102025-12-16T03:09:11.216ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "eae44e5c-4223-4bcf-be2a-19143f500fd6", "content-length": "166", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186112025-12-16T03:09:11.216ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
186122025-12-16T03:09:11.216ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "30a3722f-f00d-4026-aaaf-3eaf5a391a77", "content-length": "166", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186132025-12-16T03:09:11.216ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
186142025-12-16T03:09:11.216ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "31040fab-4d69-4091-8a28-14f13c00383a", "content-length": "702", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186152025-12-16T03:09:11.216ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
186162025-12-16T03:09:11.216ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ebc4868-cad9-40b2-a9f6-eef1331f15b1", "content-length": "727", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186172025-12-16T03:09:11.216ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
186182025-12-16T03:09:11.217ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "236b35fe-96c8-4e29-b0d4-4b86f419fc4b", "content-length": "729", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186192025-12-16T03:09:11.217ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
186202025-12-16T03:09:11.217ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3c939e5-ed6e-4ab6-ad6b-9736cd8e70a5", "content-length": "728", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186212025-12-16T03:09:11.217ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
186222025-12-16T03:09:11.217ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b74071e0-4cff-4fe1-93df-bc165499f9c7", "content-length": "722", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186232025-12-16T03:09:11.217ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
186242025-12-16T03:09:11.217ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "cef31831-6008-4191-bdf5-6923a324f33e", "content-length": "10", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186252025-12-16T03:09:11.217ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
186262025-12-16T03:09:11.217ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0e8976a-5b7f-445f-930c-82c3705ed4c1", "content-length": "144", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186272025-12-16T03:09:11.217ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
186282025-12-16T03:09:11.218ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "943beac2-1b81-4800-ae12-8a4821743bcb", "content-length": "147", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186292025-12-16T03:09:11.218ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
186302025-12-16T03:09:11.218ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f909ea94-074d-42d6-a941-1938339b0051", "content-length": "109", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186312025-12-16T03:09:11.218ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
186322025-12-16T03:09:11.219ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "34e9637c-2fa9-4ee3-a349-5de0bf79c0de", "content-length": "109", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186332025-12-16T03:09:11.219ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
186342025-12-16T03:09:11.219ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "33181747-634f-4af0-9949-787acbb07b5d", "content-length": "163", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186352025-12-16T03:09:11.219ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
186362025-12-16T03:09:11.219ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3c0e21c-78fa-4c0d-a0a8-ee2af5eff833", "content-length": "163", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186372025-12-16T03:09:11.219ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
186382025-12-16T03:09:11.220ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d87021f3-aff2-42b4-b155-97f7cb474a33", "content-length": "166", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186392025-12-16T03:09:11.220ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
186402025-12-16T03:09:11.220ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1519c44-7368-4dc1-9101-a9115152dbda", "content-length": "166", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186412025-12-16T03:09:11.220ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
186422025-12-16T03:09:11.220ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e100eccf-eb0c-4f52-bd9b-0654cf154e3a", "content-length": "702", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186432025-12-16T03:09:11.220ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
186442025-12-16T03:09:11.220ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "84c78dd3-a740-45b1-ba49-bd0529d557e5", "content-length": "727", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186452025-12-16T03:09:11.220ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
186462025-12-16T03:09:11.221ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e1690f7-f2a8-4ef9-a135-b0b7549762a9", "content-length": "729", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186472025-12-16T03:09:11.221ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
186482025-12-16T03:09:11.221ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7f1bde0-a106-44a3-bbf2-faf2f4c0ec2e", "content-length": "728", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186492025-12-16T03:09:11.221ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
186502025-12-16T03:09:11.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
186512025-12-16T03:09:11.222ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186522025-12-16T03:09:11.224ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
186532025-12-16T03:09:11.225ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
186542025-12-16T03:09:11.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
186552025-12-16T03:09:11.226ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186562025-12-16T03:09:11.228ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
186572025-12-16T03:09:11.228ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
186582025-12-16T03:09:11.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
186592025-12-16T03:09:11.265ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
186602025-12-16T03:09:11.265ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
186612025-12-16T03:09:11.368ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "67fe18ad-04ec-4bf0-b199-f482286914c3", "content-length": "95612", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186622025-12-16T03:09:11.370ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
186632025-12-16T03:09:11.370ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
186642025-12-16T03:09:11.370ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
186652025-12-16T03:09:11.370ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
186662025-12-16T03:09:11.370ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
186672025-12-16T03:09:11.370ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
186682025-12-16T03:09:11.371ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
186692025-12-16T03:09:11.371ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
186702025-12-16T03:09:11.371ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4691f135-a5e5-4ba5-b7ab-1fda80c4629b", "content-length": "64", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186712025-12-16T03:09:11.371ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
186722025-12-16T03:09:11.371ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8cf31d9f-2e38-4d4a-bd4b-0cec2fda840d", "content-length": "64", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186732025-12-16T03:09:11.371ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bac3bf0-d68e-4943-844e-76cc913e102a", "content-length": "64", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186742025-12-16T03:09:11.371ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc207885-b7f4-46bf-ac42-f9e12ceb43ba", "content-length": "64", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186752025-12-16T03:09:11.371ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1d740fc-6a25-4d8b-ae9c-b37968b27ecc", "content-length": "64", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186762025-12-16T03:09:11.371ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
186772025-12-16T03:09:11.371ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
186782025-12-16T03:09:11.371ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
186792025-12-16T03:09:11.371ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
186802025-12-16T03:09:11.422ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "756a4ada-99ec-48d7-98f1-149a2f750c7b", "content-length": "408996", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186812025-12-16T03:09:11.422ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "a25891e8-b1d5-4054-8540-4cf17acaa6dc", "content-length": "409655", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186822025-12-16T03:09:11.424ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "653e15c2-dbed-468b-b73b-80c01bebc1e5", "content-length": "409506", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186832025-12-16T03:09:11.427ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa0242b7-aac8-4d52-8042-1cafd8e17acc", "content-length": "438132", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186842025-12-16T03:09:11.428ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "8689450d-3602-47e7-841b-49d56afe625d", "content-length": "407660", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186852025-12-16T03:09:11.431ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
186862025-12-16T03:09:11.431ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
186872025-12-16T03:09:11.431ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
186882025-12-16T03:09:11.431ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
186892025-12-16T03:09:11.431ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
186902025-12-16T03:09:11.468ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c (omicron_zone)
186912025-12-16T03:09:11.468ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
186922025-12-16T03:09:11.473ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f4d1421-b764-4a1a-ab41-da14ff3b8b78", "content-length": "123", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186932025-12-16T03:09:11.474ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
186942025-12-16T03:09:11.588ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
186952025-12-16T03:09:11.588ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
186962025-12-16T03:09:11.589ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf2512dc-431a-4fa2-86a9-fcd09fc9de00", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
186972025-12-16T03:09:11.589ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
186982025-12-16T03:09:11.591ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
186992025-12-16T03:09:11.591ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
187002025-12-16T03:09:11.591ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab0184a8-9558-44db-ac85-cf25eb1ea56a", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
187012025-12-16T03:09:11.592ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
187022025-12-16T03:09:11.593ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
187032025-12-16T03:09:11.593ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
187042025-12-16T03:09:11.593ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "29bd752c-9380-4a14-aeb6-bb73b7ce5441", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:09:11 GMT"} })
187052025-12-16T03:09:11.593ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
187062025-12-16T03:09:11.594ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
187072025-12-16T03:09:11.594ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
187082025-12-16T03:09:11.596ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
187092025-12-16T03:09:11.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
187102025-12-16T03:09:11.598ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187112025-12-16T03:09:11.600ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
187122025-12-16T03:09:11.600ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
187132025-12-16T03:09:11.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
187142025-12-16T03:09:11.793ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = c98952ab-e784-47f5-b72e-5889b99827e8 file = nexus/db-queries/src/db/datastore/inventory.rs:1613
187152025-12-16T03:09:11.793ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = c98952ab-e784-47f5-b72e-5889b99827e8 time_started = 2025-12-16 03:09:11.213419 UTC
187162025-12-16T03:09:11.793ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 54 reason = Dependency
187172025-12-16T03:09:11.793ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 37 reason = Dependency
187182025-12-16T03:09:11.793ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
187192025-12-16T03:09:11.793ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
187202025-12-16T03:09:11.793ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 15.16\u{b5}s iteration = 37 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
187212025-12-16T03:09:11.793ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.040532493s iteration = 13 status = {"collection_id":"c98952ab-e784-47f5-b72e-5889b99827e8","time_done":"2025-12-16 03:09:11.594428 UTC","time_started":"2025-12-16 03:09:11.213419 UTC"}
187222025-12-16T03:09:11.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
187232025-12-16T03:09:11.795ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187242025-12-16T03:09:11.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
187252025-12-16T03:09:11.798ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
187262025-12-16T03:09:11.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
187272025-12-16T03:09:11.881ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
187282025-12-16T03:09:11.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
187292025-12-16T03:09:11.928ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 27 reason = Dependency
187302025-12-16T03:09:11.928ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 134.643617ms iteration = 54 status = {"Loaded":{"collection_id":"a51e37d5-b58a-4ed9-b46c-46ad89c1f4ad","time_loaded":"2025-12-16T03:09:11.793736990Z","time_started":"2025-12-16T03:09:11.258385Z"}}
187312025-12-16T03:09:11.928ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 38 reason = Dependency
187322025-12-16T03:09:11.928ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
187332025-12-16T03:09:11.928ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 35 reason = Dependency
187342025-12-16T03:09:11.928ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 6.92\u{b5}s iteration = 38 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
187352025-12-16T03:09:11.928ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187362025-12-16T03:09:11.928ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 17 reason = Dependency
187372025-12-16T03:09:11.928ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
187382025-12-16T03:09:11.928ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
187392025-12-16T03:09:11.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187402025-12-16T03:09:11.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
187412025-12-16T03:09:11.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
187422025-12-16T03:09:11.929ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187432025-12-16T03:09:11.930ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187442025-12-16T03:09:11.930ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187452025-12-16T03:09:11.932ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
187462025-12-16T03:09:11.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187472025-12-16T03:09:11.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187482025-12-16T03:09:11.932ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
187492025-12-16T03:09:11.932ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
187502025-12-16T03:09:11.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
187512025-12-16T03:09:11.932ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
187522025-12-16T03:09:11.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
187532025-12-16T03:09:11.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
187542025-12-16T03:09:11.934ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187552025-12-16T03:09:11.934ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187562025-12-16T03:09:11.935ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
187572025-12-16T03:09:11.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
187582025-12-16T03:09:11.936ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
187592025-12-16T03:09:11.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
187602025-12-16T03:09:11.936ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187612025-12-16T03:09:11.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
187622025-12-16T03:09:11.936ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
187632025-12-16T03:09:11.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187642025-12-16T03:09:11.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187652025-12-16T03:09:11.937ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
187662025-12-16T03:09:11.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
187672025-12-16T03:09:11.938ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
187682025-12-16T03:09:11.938ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
187692025-12-16T03:09:11.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
187702025-12-16T03:09:11.939ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187712025-12-16T03:09:11.942ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
187722025-12-16T03:09:11.942ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
187732025-12-16T03:09:11.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
187742025-12-16T03:09:11.944ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187752025-12-16T03:09:11.946ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
187762025-12-16T03:09:11.946ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
187772025-12-16T03:09:11.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
187782025-12-16T03:09:11.947ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
187792025-12-16T03:09:11.947ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
187802025-12-16T03:09:11.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
187812025-12-16T03:09:11.949ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187822025-12-16T03:09:11.953ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
187832025-12-16T03:09:11.953ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
187842025-12-16T03:09:11.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
187852025-12-16T03:09:11.955ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187862025-12-16T03:09:11.957ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
187872025-12-16T03:09:11.957ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
187882025-12-16T03:09:11.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
187892025-12-16T03:09:11.959ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
187902025-12-16T03:09:11.959ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 31.115884ms iteration = 17 status = {"blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","inventory_collection_id":"a51e37d5-b58a-4ed9-b46c-46ad89c1f4ad","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
187912025-12-16T03:09:12.029ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187922025-12-16T03:09:12.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187932025-12-16T03:09:12.031ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187942025-12-16T03:09:12.033ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
187952025-12-16T03:09:12.033ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187962025-12-16T03:09:12.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187972025-12-16T03:09:12.035ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187982025-12-16T03:09:12.037ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
187992025-12-16T03:09:12.037ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188002025-12-16T03:09:12.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188012025-12-16T03:09:12.055ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 127.504995ms iteration = 35 status = {"physical_disks_added":0}
188022025-12-16T03:09:12.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188032025-12-16T03:09:12.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188042025-12-16T03:09:12.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188052025-12-16T03:09:12.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188062025-12-16T03:09:12.083ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188072025-12-16T03:09:12.086ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
188082025-12-16T03:09:12.086ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188092025-12-16T03:09:12.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188102025-12-16T03:09:12.088ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188112025-12-16T03:09:12.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188122025-12-16T03:09:12.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188132025-12-16T03:09:12.093ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
188142025-12-16T03:09:12.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188152025-12-16T03:09:12.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188162025-12-16T03:09:12.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188172025-12-16T03:09:12.097ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
188182025-12-16T03:09:12.097ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188192025-12-16T03:09:12.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188202025-12-16T03:09:12.100ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188212025-12-16T03:09:12.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188222025-12-16T03:09:12.101ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188232025-12-16T03:09:12.104ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
188242025-12-16T03:09:12.104ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188252025-12-16T03:09:12.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188262025-12-16T03:09:12.106ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188272025-12-16T03:09:12.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188282025-12-16T03:09:12.107ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188292025-12-16T03:09:12.110ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
188302025-12-16T03:09:12.110ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188312025-12-16T03:09:12.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188322025-12-16T03:09:12.113ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188332025-12-16T03:09:12.113ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188342025-12-16T03:09:12.115ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188352025-12-16T03:09:12.117ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
188362025-12-16T03:09:12.117ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188372025-12-16T03:09:12.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188382025-12-16T03:09:12.135ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188392025-12-16T03:09:12.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188402025-12-16T03:09:12.137ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188412025-12-16T03:09:12.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188422025-12-16T03:09:12.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188432025-12-16T03:09:12.141ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
188442025-12-16T03:09:12.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188452025-12-16T03:09:12.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188462025-12-16T03:09:12.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188472025-12-16T03:09:12.146ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
188482025-12-16T03:09:12.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188492025-12-16T03:09:12.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188502025-12-16T03:09:12.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188512025-12-16T03:09:12.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188522025-12-16T03:09:12.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188532025-12-16T03:09:12.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188542025-12-16T03:09:12.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188552025-12-16T03:09:12.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
188562025-12-16T03:09:12.159ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188572025-12-16T03:09:12.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188582025-12-16T03:09:12.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188592025-12-16T03:09:12.164ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
188602025-12-16T03:09:12.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188612025-12-16T03:09:12.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188622025-12-16T03:09:12.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188632025-12-16T03:09:12.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188642025-12-16T03:09:12.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188652025-12-16T03:09:12.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188662025-12-16T03:09:12.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188672025-12-16T03:09:12.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
188682025-12-16T03:09:12.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188692025-12-16T03:09:12.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188702025-12-16T03:09:12.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188712025-12-16T03:09:12.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188722025-12-16T03:09:12.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188732025-12-16T03:09:12.179ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
188742025-12-16T03:09:12.180ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188752025-12-16T03:09:12.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188762025-12-16T03:09:12.182ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188772025-12-16T03:09:12.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188782025-12-16T03:09:12.184ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188792025-12-16T03:09:12.187ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
188802025-12-16T03:09:12.187ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188812025-12-16T03:09:12.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188822025-12-16T03:09:12.188ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188832025-12-16T03:09:12.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188842025-12-16T03:09:12.190ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188852025-12-16T03:09:12.193ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
188862025-12-16T03:09:12.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188872025-12-16T03:09:12.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188882025-12-16T03:09:12.195ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188892025-12-16T03:09:12.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188902025-12-16T03:09:12.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188912025-12-16T03:09:12.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
188922025-12-16T03:09:12.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188932025-12-16T03:09:12.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188942025-12-16T03:09:12.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188952025-12-16T03:09:12.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
188962025-12-16T03:09:12.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188972025-12-16T03:09:12.209ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
188982025-12-16T03:09:12.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
188992025-12-16T03:09:12.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
189002025-12-16T03:09:12.247ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
189012025-12-16T03:09:12.248ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
189022025-12-16T03:09:12.248ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = cc39429e-1963-48d0-b0ff-e97551d07b62 parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
189032025-12-16T03:09:12.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
189042025-12-16T03:09:12.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
189052025-12-16T03:09:12.249ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189062025-12-16T03:09:12.252ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
189072025-12-16T03:09:12.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
189082025-12-16T03:09:12.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
189092025-12-16T03:09:12.255ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
189102025-12-16T03:09:12.255ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
189112025-12-16T03:09:12.255ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 326.79131ms iteration = 27 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
189122025-12-16T03:09:14.317ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 122 reason = Timeout
189132025-12-16T03:09:14.317ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
189142025-12-16T03:09:14.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
189152025-12-16T03:09:14.320ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189162025-12-16T03:09:14.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
189172025-12-16T03:09:14.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
189182025-12-16T03:09:14.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
189192025-12-16T03:09:14.325ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
189202025-12-16T03:09:14.325ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.902661ms iteration = 122 status = {"config_updated":false}
189212025-12-16T03:09:19.318ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 62 reason = Timeout
189222025-12-16T03:09:19.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 62 reason = Timeout
189232025-12-16T03:09:19.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
189242025-12-16T03:09:19.318ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 123 reason = Timeout
189252025-12-16T03:09:19.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
189262025-12-16T03:09:19.318ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
189272025-12-16T03:09:19.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
189282025-12-16T03:09:19.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
189292025-12-16T03:09:19.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
189302025-12-16T03:09:19.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189312025-12-16T03:09:19.320ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189322025-12-16T03:09:19.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
189332025-12-16T03:09:19.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
189342025-12-16T03:09:19.324ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
189352025-12-16T03:09:19.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
189362025-12-16T03:09:19.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
189372025-12-16T03:09:19.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
189382025-12-16T03:09:19.325ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
189392025-12-16T03:09:19.325ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.251692ms iteration = 123 status = {"config_updated":false}
189402025-12-16T03:09:19.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
189412025-12-16T03:09:19.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
189422025-12-16T03:09:19.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189432025-12-16T03:09:19.331ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
189442025-12-16T03:09:19.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
189452025-12-16T03:09:19.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
189462025-12-16T03:09:19.349ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
189472025-12-16T03:09:19.349ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 31.405188ms iteration = 62 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
189482025-12-16T03:09:19.359ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
189492025-12-16T03:09:19.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
189502025-12-16T03:09:19.397ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 78.573745ms iteration = 62 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
189512025-12-16T03:09:21.068ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 286 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = e6722b34-07ad-4769-9be7-aed1a938b27f response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
189522025-12-16T03:09:24.317ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 42 reason = Timeout
189532025-12-16T03:09:24.317ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 55 reason = Timeout
189542025-12-16T03:09:24.317ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
189552025-12-16T03:09:24.317ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
189562025-12-16T03:09:24.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
189572025-12-16T03:09:24.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
189582025-12-16T03:09:24.318ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 42 reason = Timeout
189592025-12-16T03:09:24.318ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 124 reason = Timeout
189602025-12-16T03:09:24.318ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
189612025-12-16T03:09:24.318ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
189622025-12-16T03:09:24.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
189632025-12-16T03:09:24.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
189642025-12-16T03:09:24.319ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
189652025-12-16T03:09:24.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
189662025-12-16T03:09:24.319ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189672025-12-16T03:09:24.319ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189682025-12-16T03:09:24.320ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189692025-12-16T03:09:24.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189702025-12-16T03:09:24.322ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
189712025-12-16T03:09:24.322ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
189722025-12-16T03:09:24.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
189732025-12-16T03:09:24.323ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
189742025-12-16T03:09:24.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
189752025-12-16T03:09:24.323ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
189762025-12-16T03:09:24.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
189772025-12-16T03:09:24.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
189782025-12-16T03:09:24.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
189792025-12-16T03:09:24.324ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.501603ms iteration = 42 status = "NoSitrep"
189802025-12-16T03:09:24.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
189812025-12-16T03:09:24.325ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = a51e37d5-b58a-4ed9-b46c-46ad89c1f4ad original_time_started = 2025-12-16 03:09:11.258385 UTC
189822025-12-16T03:09:24.325ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.783629ms iteration = 55 status = {"Loaded":{"collection_id":"a51e37d5-b58a-4ed9-b46c-46ad89c1f4ad","time_loaded":"2025-12-16T03:09:24.317572065Z","time_started":"2025-12-16T03:09:11.258385Z"}}
189832025-12-16T03:09:24.325ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
189842025-12-16T03:09:24.325ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.215421ms iteration = 124 status = {"config_updated":false}
189852025-12-16T03:09:24.325ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
189862025-12-16T03:09:24.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
189872025-12-16T03:09:24.326ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
189882025-12-16T03:09:24.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
189892025-12-16T03:09:24.327ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189902025-12-16T03:09:24.329ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
189912025-12-16T03:09:24.330ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
189922025-12-16T03:09:24.330ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 11.87622ms iteration = 42 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
189932025-12-16T03:09:29.318ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 63 reason = Timeout
189942025-12-16T03:09:29.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 63 reason = Timeout
189952025-12-16T03:09:29.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
189962025-12-16T03:09:29.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
189972025-12-16T03:09:29.318ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 125 reason = Timeout
189982025-12-16T03:09:29.318ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
189992025-12-16T03:09:29.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
190002025-12-16T03:09:29.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
190012025-12-16T03:09:29.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
190022025-12-16T03:09:29.320ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190032025-12-16T03:09:29.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190042025-12-16T03:09:29.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
190052025-12-16T03:09:29.324ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
190062025-12-16T03:09:29.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
190072025-12-16T03:09:29.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
190082025-12-16T03:09:29.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
190092025-12-16T03:09:29.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
190102025-12-16T03:09:29.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
190112025-12-16T03:09:29.326ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
190122025-12-16T03:09:29.326ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.190164ms iteration = 125 status = {"config_updated":false}
190132025-12-16T03:09:29.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
190142025-12-16T03:09:29.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190152025-12-16T03:09:29.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
190162025-12-16T03:09:29.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
190172025-12-16T03:09:29.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
190182025-12-16T03:09:29.342ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
190192025-12-16T03:09:29.342ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.563112ms iteration = 63 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
190202025-12-16T03:09:29.356ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
190212025-12-16T03:09:29.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
190222025-12-16T03:09:29.357ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 39.302237ms iteration = 63 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
190232025-12-16T03:09:31.068ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 310 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 0082907a-3d11-4ada-b141-2e51656c8a47 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
190242025-12-16T03:09:34.318ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 126 reason = Timeout
190252025-12-16T03:09:34.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
190262025-12-16T03:09:34.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
190272025-12-16T03:09:34.321ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190282025-12-16T03:09:34.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
190292025-12-16T03:09:34.324ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
190302025-12-16T03:09:34.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
190312025-12-16T03:09:34.326ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
190322025-12-16T03:09:34.326ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.1196ms iteration = 126 status = {"config_updated":false}
190332025-12-16T03:09:39.176ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
190342025-12-16T03:09:39.176ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 22 reason = Timeout
190352025-12-16T03:09:39.176ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
190362025-12-16T03:09:39.176ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
190372025-12-16T03:09:39.176ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
190382025-12-16T03:09:39.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
190392025-12-16T03:09:39.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
190402025-12-16T03:09:39.179ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1586, tv_nsec: 788143375 } })
190412025-12-16T03:09:39.179ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1586, tv_nsec: 788302517 } })
190422025-12-16T03:09:39.180ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
190432025-12-16T03:09:39.180ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1586, tv_nsec: 788989086 } })
190442025-12-16T03:09:39.180ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.811648ms iteration = 22 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
190452025-12-16T03:09:39.254ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
190462025-12-16T03:09:39.255ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 78.798268ms iteration = 22 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
190472025-12-16T03:09:39.266ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
190482025-12-16T03:09:39.266ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
190492025-12-16T03:09:39.266ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7cda8680-4b09-4914-a535-154ffbe09d46", "content-length": "26", "date": "Tue, 16 Dec 2025 03:09:39 GMT"} })
190502025-12-16T03:09:39.266ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
190512025-12-16T03:09:39.268ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
190522025-12-16T03:09:39.268ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1586, tv_nsec: 877325076 } })
190532025-12-16T03:09:39.308ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 131.79316ms iteration = 22 status = {}
190542025-12-16T03:09:39.317ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
190552025-12-16T03:09:39.317ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 43 reason = Timeout
190562025-12-16T03:09:39.317ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 56 reason = Timeout
190572025-12-16T03:09:39.317ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
190582025-12-16T03:09:39.317ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
190592025-12-16T03:09:39.317ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
190602025-12-16T03:09:39.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
190612025-12-16T03:09:39.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
190622025-12-16T03:09:39.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
190632025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 22 reason = Timeout
190642025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 22 reason = Timeout
190652025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
190662025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 22 reason = Timeout
190672025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 15.441\u{b5}s iteration = 22 status = {"disabled":true,"errors":[],"sps":[]}
190682025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
190692025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 43 reason = Timeout
190702025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
190712025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
190722025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
190732025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
190742025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 22 reason = Timeout
190752025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
190762025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
190772025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 33 reason = Timeout
190782025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 22 reason = Timeout
190792025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
190802025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
190812025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
190822025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
190832025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 22 reason = Timeout
190842025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 36 reason = Timeout
190852025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
190862025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
190872025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 64 reason = Timeout
190882025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
190892025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 127 reason = Timeout
190902025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
190912025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
190922025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 22 reason = Timeout
190932025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
190942025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
190952025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
190962025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 22 reason = Timeout
190972025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
190982025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 33 reason = Timeout
190992025-12-16T03:09:39.318ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
191002025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
191012025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
191022025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
191032025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
191042025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
191052025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
191062025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
191072025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
191082025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
191092025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
191102025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
191112025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
191122025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
191132025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
191142025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
191152025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
191162025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
191172025-12-16T03:09:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
191182025-12-16T03:09:39.319ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191192025-12-16T03:09:39.319ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191202025-12-16T03:09:39.319ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
191212025-12-16T03:09:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
191222025-12-16T03:09:39.320ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191232025-12-16T03:09:39.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191242025-12-16T03:09:39.320ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
191252025-12-16T03:09:39.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
191262025-12-16T03:09:39.322ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 4.280234ms iteration = 22 status = {"errors":[],"finish_invoked_ok":[]}
191272025-12-16T03:09:39.322ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191282025-12-16T03:09:39.324ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
191292025-12-16T03:09:39.324ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
191302025-12-16T03:09:39.325ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
191312025-12-16T03:09:39.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
191322025-12-16T03:09:39.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
191332025-12-16T03:09:39.325ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
191342025-12-16T03:09:39.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
191352025-12-16T03:09:39.325ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
191362025-12-16T03:09:39.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
191372025-12-16T03:09:39.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
191382025-12-16T03:09:39.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
191392025-12-16T03:09:39.325ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
191402025-12-16T03:09:39.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
191412025-12-16T03:09:39.328ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = a51e37d5-b58a-4ed9-b46c-46ad89c1f4ad original_time_started = 2025-12-16 03:09:11.258385 UTC
191422025-12-16T03:09:39.328ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 11.155911ms iteration = 56 status = {"Loaded":{"collection_id":"a51e37d5-b58a-4ed9-b46c-46ad89c1f4ad","time_loaded":"2025-12-16T03:09:39.317599401Z","time_started":"2025-12-16T03:09:11.258385Z"}}
191432025-12-16T03:09:39.329ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1586, tv_nsec: 938019334 } })
191442025-12-16T03:09:39.329ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
191452025-12-16T03:09:39.330ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1586, tv_nsec: 938767254 } })
191462025-12-16T03:09:39.333ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 16.340547ms iteration = 43 status = "NoSitrep"
191472025-12-16T03:09:39.335ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
191482025-12-16T03:09:39.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
191492025-12-16T03:09:39.336ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
191502025-12-16T03:09:39.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
191512025-12-16T03:09:39.339ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191522025-12-16T03:09:39.346ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
191532025-12-16T03:09:39.347ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
191542025-12-16T03:09:39.347ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 29.413672ms iteration = 22 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
191552025-12-16T03:09:39.347ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
191562025-12-16T03:09:39.347ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 29.453124ms iteration = 43 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
191572025-12-16T03:09:39.378ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
191582025-12-16T03:09:39.379ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
191592025-12-16T03:09:39.379ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d010a9a8-c667-4a93-9904-8f609d5b1825", "content-length": "26", "date": "Tue, 16 Dec 2025 03:09:39 GMT"} })
191602025-12-16T03:09:39.379ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
191612025-12-16T03:09:39.380ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191622025-12-16T03:09:39.381ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191632025-12-16T03:09:39.381ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
191642025-12-16T03:09:39.382ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
191652025-12-16T03:09:39.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
191662025-12-16T03:09:39.384ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
191672025-12-16T03:09:39.384ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
191682025-12-16T03:09:39.384ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
191692025-12-16T03:09:39.384ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
191702025-12-16T03:09:39.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
191712025-12-16T03:09:39.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
191722025-12-16T03:09:39.385ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191732025-12-16T03:09:39.385ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 67.258193ms iteration = 22 status = {"errors":[],"requests_created_ok":[]}
191742025-12-16T03:09:39.385ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
191752025-12-16T03:09:39.386ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191762025-12-16T03:09:39.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
191772025-12-16T03:09:39.387ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
191782025-12-16T03:09:39.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
191792025-12-16T03:09:39.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191802025-12-16T03:09:39.388ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 69.845735ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
191812025-12-16T03:09:39.388ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
191822025-12-16T03:09:39.388ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
191832025-12-16T03:09:39.388ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
191842025-12-16T03:09:39.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
191852025-12-16T03:09:39.389ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
191862025-12-16T03:09:39.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
191872025-12-16T03:09:39.389ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191882025-12-16T03:09:39.390ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
191892025-12-16T03:09:39.390ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
191902025-12-16T03:09:39.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
191912025-12-16T03:09:39.390ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
191922025-12-16T03:09:39.390ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
191932025-12-16T03:09:39.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
191942025-12-16T03:09:39.391ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
191952025-12-16T03:09:39.391ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 73.360129ms iteration = 22 status = {"errors":[],"garbage_collect_requested":[]}
191962025-12-16T03:09:39.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
191972025-12-16T03:09:39.391ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191982025-12-16T03:09:39.392ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
191992025-12-16T03:09:39.392ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 74.15234ms iteration = 127 status = {"config_updated":false}
192002025-12-16T03:09:39.393ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 75.254583ms iteration = 22 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
192012025-12-16T03:09:39.393ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
192022025-12-16T03:09:39.393ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
192032025-12-16T03:09:39.393ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
192042025-12-16T03:09:39.393ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
192052025-12-16T03:09:39.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
192062025-12-16T03:09:39.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
192072025-12-16T03:09:39.394ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
192082025-12-16T03:09:39.394ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
192092025-12-16T03:09:39.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
192102025-12-16T03:09:39.396ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
192112025-12-16T03:09:39.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
192122025-12-16T03:09:39.397ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
192132025-12-16T03:09:39.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
192142025-12-16T03:09:39.398ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192152025-12-16T03:09:39.401ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192162025-12-16T03:09:39.402ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
192172025-12-16T03:09:39.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
192182025-12-16T03:09:39.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
192192025-12-16T03:09:39.407ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
192202025-12-16T03:09:39.407ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
192212025-12-16T03:09:39.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
192222025-12-16T03:09:39.410ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 92.593224ms iteration = 22 status = 0
192232025-12-16T03:09:39.422ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1587, tv_nsec: 30682769 } })
192242025-12-16T03:09:39.424ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
192252025-12-16T03:09:39.424ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 106.258766ms iteration = 64 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
192262025-12-16T03:09:39.443ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
192272025-12-16T03:09:39.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
192282025-12-16T03:09:39.446ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192292025-12-16T03:09:39.449ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
192302025-12-16T03:09:39.449ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
192312025-12-16T03:09:39.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
192322025-12-16T03:09:39.450ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 132.179904ms iteration = 64 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
192332025-12-16T03:09:39.455ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
192342025-12-16T03:09:39.455ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
192352025-12-16T03:09:39.455ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
192362025-12-16T03:09:39.455ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 137.025256ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
192372025-12-16T03:09:39.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
192382025-12-16T03:09:39.477ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1587, tv_nsec: 85953528 } })
192392025-12-16T03:09:39.506ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
192402025-12-16T03:09:39.506ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 188.309885ms iteration = 33 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
192412025-12-16T03:09:39.518ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
192422025-12-16T03:09:39.518ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
192432025-12-16T03:09:39.518ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 200.345988ms iteration = 22 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
192442025-12-16T03:09:39.520ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
192452025-12-16T03:09:39.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
192462025-12-16T03:09:39.534ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
192472025-12-16T03:09:39.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
192482025-12-16T03:09:39.537ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
192492025-12-16T03:09:39.537ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
192502025-12-16T03:09:39.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
192512025-12-16T03:09:39.550ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 231.813277ms iteration = 36 status = {"physical_disks_added":0}
192522025-12-16T03:09:39.551ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
192532025-12-16T03:09:39.551ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
192542025-12-16T03:09:39.552ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
192552025-12-16T03:09:39.552ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
192562025-12-16T03:09:39.552ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
192572025-12-16T03:09:39.552ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
192582025-12-16T03:09:39.552ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
192592025-12-16T03:09:39.552ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
192602025-12-16T03:09:39.553ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
192612025-12-16T03:09:39.553ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
192622025-12-16T03:09:39.553ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
192632025-12-16T03:09:39.553ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
192642025-12-16T03:09:39.553ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
192652025-12-16T03:09:39.553ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
192662025-12-16T03:09:39.553ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce88e67b-b2c1-41b5-bd35-5a5fbe01ed34", "content-length": "372", "date": "Tue, 16 Dec 2025 03:09:39 GMT"} })
192672025-12-16T03:09:39.553ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
192682025-12-16T03:09:39.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
192692025-12-16T03:09:39.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
192702025-12-16T03:09:39.556ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
192712025-12-16T03:09:39.556ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
192722025-12-16T03:09:39.569ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
192732025-12-16T03:09:39.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
192742025-12-16T03:09:39.571ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192752025-12-16T03:09:39.573ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
192762025-12-16T03:09:39.573ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
192772025-12-16T03:09:39.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
192782025-12-16T03:09:39.575ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192792025-12-16T03:09:39.577ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
192802025-12-16T03:09:39.577ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
192812025-12-16T03:09:39.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
192822025-12-16T03:09:39.612ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
192832025-12-16T03:09:39.612ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
192842025-12-16T03:09:39.612ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
192852025-12-16T03:09:39.613ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "44807acf-8b14-453a-bdb4-cf1b966c471d", "date": "Tue, 16 Dec 2025 03:09:39 GMT"} })
192862025-12-16T03:09:39.613ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
192872025-12-16T03:09:39.613ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9a51075-181b-481e-9478-ec2788627d76", "content-length": "691", "date": "Tue, 16 Dec 2025 03:09:39 GMT"} })
192882025-12-16T03:09:39.613ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
192892025-12-16T03:09:39.613ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
192902025-12-16T03:09:39.613ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
192912025-12-16T03:09:39.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
192922025-12-16T03:09:39.615ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 297.026254ms iteration = 22 status = {}
192932025-12-16T03:09:39.615ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
192942025-12-16T03:09:39.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
192952025-12-16T03:09:39.618ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
192962025-12-16T03:09:39.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
192972025-12-16T03:09:39.622ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
192982025-12-16T03:09:39.622ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
192992025-12-16T03:09:39.623ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "9a683f07-f3bb-44b6-b542-57d6488b9e19", "date": "Tue, 16 Dec 2025 03:09:39 GMT"} })
193002025-12-16T03:09:39.625ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 306.783597ms iteration = 33 status = {}
193012025-12-16T03:09:39.766ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "1b883e95-9e26-4a0f-9c18-40bd869ddcb7", "date": "Tue, 16 Dec 2025 03:09:39 GMT"} })
193022025-12-16T03:09:39.766ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
193032025-12-16T03:09:39.766ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
193042025-12-16T03:09:39.766ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "94f46170-35ef-48e5-9cb0-a071cf3e0418", "content-length": "682", "date": "Tue, 16 Dec 2025 03:09:39 GMT"} })
193052025-12-16T03:09:39.767ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
193062025-12-16T03:09:39.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
193072025-12-16T03:09:39.770ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
193082025-12-16T03:09:39.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
193092025-12-16T03:09:39.773ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
193102025-12-16T03:09:39.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
193112025-12-16T03:09:39.775ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
193122025-12-16T03:09:39.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
193132025-12-16T03:09:39.782ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 465.354426ms iteration = 22 status = {}
193142025-12-16T03:09:41.068ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 283 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = d2c74257-6f3f-46f6-9136-ce940a0c76be response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
193152025-12-16T03:09:44.319ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 128 reason = Timeout
193162025-12-16T03:09:44.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
193172025-12-16T03:09:44.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
193182025-12-16T03:09:44.321ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193192025-12-16T03:09:44.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
193202025-12-16T03:09:44.325ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
193212025-12-16T03:09:44.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
193222025-12-16T03:09:44.327ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
193232025-12-16T03:09:44.327ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.331936ms iteration = 128 status = {"config_updated":false}
193242025-12-16T03:09:49.318ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
193252025-12-16T03:09:49.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 65 reason = Timeout
193262025-12-16T03:09:49.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
193272025-12-16T03:09:49.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
193282025-12-16T03:09:49.318ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 129 reason = Timeout
193292025-12-16T03:09:49.318ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
193302025-12-16T03:09:49.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
193312025-12-16T03:09:49.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
193322025-12-16T03:09:49.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
193332025-12-16T03:09:49.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193342025-12-16T03:09:49.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
193352025-12-16T03:09:49.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
193362025-12-16T03:09:49.322ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.178583ms iteration = 65 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
193372025-12-16T03:09:49.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
193382025-12-16T03:09:49.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
193392025-12-16T03:09:49.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
193402025-12-16T03:09:49.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
193412025-12-16T03:09:49.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
193422025-12-16T03:09:49.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193432025-12-16T03:09:49.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
193442025-12-16T03:09:49.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
193452025-12-16T03:09:49.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
193462025-12-16T03:09:49.347ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
193472025-12-16T03:09:49.347ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.920496ms iteration = 65 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
193482025-12-16T03:09:49.357ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193492025-12-16T03:09:49.360ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
193502025-12-16T03:09:49.360ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
193512025-12-16T03:09:49.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
193522025-12-16T03:09:49.362ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
193532025-12-16T03:09:49.362ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 43.819425ms iteration = 129 status = {"config_updated":false}
193542025-12-16T03:09:51.062ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:59463
193552025-12-16T03:09:51.062ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59463 req_id = c2c21c32-6343-421a-b33e-0fecf56bc8c4 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending version = 1.0.0
193562025-12-16T03:09:51.062ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59463 req_id = c2c21c32-6343-421a-b33e-0fecf56bc8c4 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
193572025-12-16T03:09:51.063ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59463 req_id = c2c21c32-6343-421a-b33e-0fecf56bc8c4 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
193582025-12-16T03:09:51.066ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 3954 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59463 req_id = c2c21c32-6343-421a-b33e-0fecf56bc8c4 response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending
193592025-12-16T03:09:51.066ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59463 req_id = fe1d2814-f568-4bac-bc9f-2983636076b9 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D version = 1.0.0
193602025-12-16T03:09:51.066ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59463 req_id = fe1d2814-f568-4bac-bc9f-2983636076b9 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
193612025-12-16T03:09:51.067ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59463 req_id = fe1d2814-f568-4bac-bc9f-2983636076b9 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
193622025-12-16T03:09:51.068ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 262 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = a2bcac71-54db-4335-b505-956b86dde3c3 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
193632025-12-16T03:09:51.069ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 2949 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59463 req_id = fe1d2814-f568-4bac-bc9f-2983636076b9 response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D
193642025-12-16T03:09:54.318ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 57 reason = Timeout
193652025-12-16T03:09:54.318ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 44 reason = Timeout
193662025-12-16T03:09:54.318ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
193672025-12-16T03:09:54.318ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
193682025-12-16T03:09:54.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
193692025-12-16T03:09:54.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
193702025-12-16T03:09:54.318ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 44 reason = Timeout
193712025-12-16T03:09:54.318ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 130 reason = Timeout
193722025-12-16T03:09:54.318ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
193732025-12-16T03:09:54.318ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
193742025-12-16T03:09:54.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
193752025-12-16T03:09:54.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
193762025-12-16T03:09:54.319ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
193772025-12-16T03:09:54.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
193782025-12-16T03:09:54.320ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193792025-12-16T03:09:54.320ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193802025-12-16T03:09:54.320ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193812025-12-16T03:09:54.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193822025-12-16T03:09:54.324ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
193832025-12-16T03:09:54.324ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
193842025-12-16T03:09:54.324ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
193852025-12-16T03:09:54.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
193862025-12-16T03:09:54.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
193872025-12-16T03:09:54.325ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
193882025-12-16T03:09:54.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
193892025-12-16T03:09:54.325ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
193902025-12-16T03:09:54.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
193912025-12-16T03:09:54.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
193922025-12-16T03:09:54.326ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 7.983761ms iteration = 44 status = "NoSitrep"
193932025-12-16T03:09:54.326ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
193942025-12-16T03:09:54.326ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.693297ms iteration = 130 status = {"config_updated":false}
193952025-12-16T03:09:54.326ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = a51e37d5-b58a-4ed9-b46c-46ad89c1f4ad original_time_started = 2025-12-16 03:09:11.258385 UTC
193962025-12-16T03:09:54.326ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.976954ms iteration = 57 status = {"Loaded":{"collection_id":"a51e37d5-b58a-4ed9-b46c-46ad89c1f4ad","time_loaded":"2025-12-16T03:09:54.318013364Z","time_started":"2025-12-16T03:09:11.258385Z"}}
193972025-12-16T03:09:54.327ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
193982025-12-16T03:09:54.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
193992025-12-16T03:09:54.327ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
194002025-12-16T03:09:54.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
194012025-12-16T03:09:54.329ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
194022025-12-16T03:09:54.330ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
194032025-12-16T03:09:54.331ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
194042025-12-16T03:09:54.331ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 12.939884ms iteration = 44 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
194052025-12-16T03:09:59.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 66 reason = Timeout
194062025-12-16T03:09:59.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
194072025-12-16T03:09:59.318ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 66 reason = Timeout
194082025-12-16T03:09:59.318ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 131 reason = Timeout
194092025-12-16T03:09:59.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
194102025-12-16T03:09:59.318ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
194112025-12-16T03:09:59.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
194122025-12-16T03:09:59.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
194132025-12-16T03:09:59.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
194142025-12-16T03:09:59.320ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
194152025-12-16T03:09:59.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
194162025-12-16T03:09:59.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
194172025-12-16T03:09:59.324ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
194182025-12-16T03:09:59.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
194192025-12-16T03:09:59.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
194202025-12-16T03:09:59.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
194212025-12-16T03:09:59.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
194222025-12-16T03:09:59.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
194232025-12-16T03:09:59.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
194242025-12-16T03:09:59.329ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
194252025-12-16T03:09:59.329ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 10.772466ms iteration = 131 status = {"config_updated":false}
194262025-12-16T03:09:59.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
194272025-12-16T03:09:59.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
194282025-12-16T03:09:59.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
194292025-12-16T03:09:59.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
194302025-12-16T03:09:59.348ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
194312025-12-16T03:09:59.348ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.840748ms iteration = 66 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
194322025-12-16T03:09:59.361ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
194332025-12-16T03:09:59.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
194342025-12-16T03:09:59.400ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 81.352881ms iteration = 66 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
194352025-12-16T03:10:01.069ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 444 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 84f6601a-4672-401a-a209-61b38aed5d76 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
194362025-12-16T03:10:04.318ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 132 reason = Timeout
194372025-12-16T03:10:04.318ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
194382025-12-16T03:10:04.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
194392025-12-16T03:10:04.321ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
194402025-12-16T03:10:04.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
194412025-12-16T03:10:04.324ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
194422025-12-16T03:10:04.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
194432025-12-16T03:10:04.326ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
194442025-12-16T03:10:04.326ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.688007ms iteration = 132 status = {"config_updated":false}
194452025-12-16T03:10:09.176ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
194462025-12-16T03:10:09.176ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
194472025-12-16T03:10:09.176ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
194482025-12-16T03:10:09.176ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
194492025-12-16T03:10:09.176ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 23 reason = Timeout
194502025-12-16T03:10:09.176ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
194512025-12-16T03:10:09.176ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
194522025-12-16T03:10:09.176ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
194532025-12-16T03:10:09.176ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
194542025-12-16T03:10:09.176ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
194552025-12-16T03:10:09.176ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
194562025-12-16T03:10:09.176ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
194572025-12-16T03:10:09.176ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
194582025-12-16T03:10:09.176ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
194592025-12-16T03:10:09.176ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
194602025-12-16T03:10:09.176ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
194612025-12-16T03:10:09.176ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
194622025-12-16T03:10:09.176ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
194632025-12-16T03:10:09.176ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2b346da2-5029-4762-b024-fccd46d69b68 step component = ExternalNetworking step id = Ensure
194642025-12-16T03:10:09.176ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
194652025-12-16T03:10:09.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
194662025-12-16T03:10:09.176ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
194672025-12-16T03:10:09.176ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
194682025-12-16T03:10:09.177ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-12-16 03:00:09.176927263 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
194692025-12-16T03:10:09.177ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
194702025-12-16T03:10:09.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
194712025-12-16T03:10:09.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
194722025-12-16T03:10:09.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
194732025-12-16T03:10:09.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
194742025-12-16T03:10:09.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
194752025-12-16T03:10:09.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
194762025-12-16T03:10:09.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
194772025-12-16T03:10:09.177ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1a45cd23-3d0a-43eb-adce-62c7b712baf1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1616, tv_nsec: 785593923 } })
194782025-12-16T03:10:09.177ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("49a51a59-9919-48ae-8182-4070a8430160.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6af60386-8771-4401-9d4d-e93774e519f0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 1616, tv_nsec: 785598894 } })
194792025-12-16T03:10:09.178ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
194802025-12-16T03:10:09.178ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.604871ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
194812025-12-16T03:10:09.178ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
194822025-12-16T03:10:09.178ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.466969ms iteration = 12 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
194832025-12-16T03:10:09.178ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
194842025-12-16T03:10:09.178ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
194852025-12-16T03:10:09.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
194862025-12-16T03:10:09.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
194872025-12-16T03:10:09.179ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
194882025-12-16T03:10:09.180ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1616, tv_nsec: 788715423 } })
194892025-12-16T03:10:09.180ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-12-16 03:00:09.176927263 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
194902025-12-16T03:10:09.180ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.842469ms iteration = 12 status = {"expiration":"2025-12-16T03:00:09.176927263Z","pruned":[]}
194912025-12-16T03:10:09.181ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1616, tv_nsec: 789328341 } })
194922025-12-16T03:10:09.181ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
194932025-12-16T03:10:09.181ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
194942025-12-16T03:10:09.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
194952025-12-16T03:10:09.182ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.342728ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
194962025-12-16T03:10:09.182ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1616, tv_nsec: 790107431 } })
194972025-12-16T03:10:09.184ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
194982025-12-16T03:10:09.184ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
194992025-12-16T03:10:09.184ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
195002025-12-16T03:10:09.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
195012025-12-16T03:10:09.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
195022025-12-16T03:10:09.186ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195032025-12-16T03:10:09.187ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
195042025-12-16T03:10:09.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
195052025-12-16T03:10:09.192ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
195062025-12-16T03:10:09.217ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 28 reason = Timeout
195072025-12-16T03:10:09.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
195082025-12-16T03:10:09.218ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
195092025-12-16T03:10:09.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
195102025-12-16T03:10:09.219ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
195112025-12-16T03:10:09.219ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
195122025-12-16T03:10:09.219ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
195132025-12-16T03:10:09.219ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
195142025-12-16T03:10:09.219ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
195152025-12-16T03:10:09.219ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
195162025-12-16T03:10:09.219ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
195172025-12-16T03:10:09.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
195182025-12-16T03:10:09.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
195192025-12-16T03:10:09.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
195202025-12-16T03:10:09.221ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195212025-12-16T03:10:09.221ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
195222025-12-16T03:10:09.222ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195232025-12-16T03:10:09.224ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
195242025-12-16T03:10:09.224ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
195252025-12-16T03:10:09.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
195262025-12-16T03:10:09.226ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
195272025-12-16T03:10:09.226ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
195282025-12-16T03:10:09.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
195292025-12-16T03:10:09.229ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
195302025-12-16T03:10:09.231ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
195312025-12-16T03:10:09.236ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 39 reason = Timeout
195322025-12-16T03:10:09.236ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
195332025-12-16T03:10:09.236ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 12 reason = Timeout
195342025-12-16T03:10:09.236ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
195352025-12-16T03:10:09.236ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
195362025-12-16T03:10:09.236ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 33.19\u{b5}s iteration = 39 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
195372025-12-16T03:10:09.236ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
195382025-12-16T03:10:09.236ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 12 reason = Timeout
195392025-12-16T03:10:09.236ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 12 reason = Timeout
195402025-12-16T03:10:09.236ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
195412025-12-16T03:10:09.236ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
195422025-12-16T03:10:09.236ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
195432025-12-16T03:10:09.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
195442025-12-16T03:10:09.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
195452025-12-16T03:10:09.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
195462025-12-16T03:10:09.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
195472025-12-16T03:10:09.237ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195482025-12-16T03:10:09.239ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 3.514685ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
195492025-12-16T03:10:09.240ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
195502025-12-16T03:10:09.242ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
195512025-12-16T03:10:09.242ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
195522025-12-16T03:10:09.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
195532025-12-16T03:10:09.242ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
195542025-12-16T03:10:09.243ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
195552025-12-16T03:10:09.243ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 6.998029ms iteration = 12 status = {"by_rx":{},"error":null}
195562025-12-16T03:10:09.246ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
195572025-12-16T03:10:09.249ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
195582025-12-16T03:10:09.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
195592025-12-16T03:10:09.262ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
195602025-12-16T03:10:09.265ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195612025-12-16T03:10:09.266ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 29.957389ms iteration = 12 status = {"dispatched":[],"errors":[],"glob_version":"212.0.0","globs_reprocessed":{},"no_receivers":[]}
195622025-12-16T03:10:09.272ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
195632025-12-16T03:10:09.272ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
195642025-12-16T03:10:09.274ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
195652025-12-16T03:10:09.278ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
195662025-12-16T03:10:09.279ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
195672025-12-16T03:10:09.280ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2b346da2-5029-4762-b024-fccd46d69b68 step component = SledList step id = Fetch
195682025-12-16T03:10:09.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
195692025-12-16T03:10:09.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
195702025-12-16T03:10:09.286ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
195712025-12-16T03:10:09.286ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
195722025-12-16T03:10:09.286ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
195732025-12-16T03:10:09.286ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
195742025-12-16T03:10:09.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
195752025-12-16T03:10:09.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195762025-12-16T03:10:09.289ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195772025-12-16T03:10:09.291ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
195782025-12-16T03:10:09.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
195792025-12-16T03:10:09.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
195802025-12-16T03:10:09.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
195812025-12-16T03:10:09.296ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
195822025-12-16T03:10:09.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
195832025-12-16T03:10:09.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195842025-12-16T03:10:09.301ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195852025-12-16T03:10:09.304ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
195862025-12-16T03:10:09.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
195872025-12-16T03:10:09.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
195882025-12-16T03:10:09.309ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
195892025-12-16T03:10:09.311ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
195902025-12-16T03:10:09.311ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
195912025-12-16T03:10:09.311ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
195922025-12-16T03:10:09.311ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
195932025-12-16T03:10:09.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
195942025-12-16T03:10:09.314ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195952025-12-16T03:10:09.315ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195962025-12-16T03:10:09.317ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
195972025-12-16T03:10:09.317ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 45 reason = Timeout
195982025-12-16T03:10:09.317ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
195992025-12-16T03:10:09.317ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
196002025-12-16T03:10:09.317ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 58 reason = Timeout
196012025-12-16T03:10:09.317ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
196022025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
196032025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
196042025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
196052025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 23 reason = Timeout
196062025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 17.791\u{b5}s iteration = 23 status = {"disabled":true,"errors":[],"sps":[]}
196072025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
196082025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
196092025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 45 reason = Timeout
196102025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 34 reason = Timeout
196112025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
196122025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
196132025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
196142025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
196152025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
196162025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
196172025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
196182025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
196192025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 37 reason = Timeout
196202025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
196212025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
196222025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 34 reason = Timeout
196232025-12-16T03:10:09.318ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
196242025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
196252025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 133 reason = Timeout
196262025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
196272025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
196282025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
196292025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
196302025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
196312025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
196322025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
196332025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 23 reason = Timeout
196342025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
196352025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
196362025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
196372025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
196382025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
196392025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
196402025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
196412025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
196422025-12-16T03:10:09.318ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
196432025-12-16T03:10:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
196442025-12-16T03:10:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
196452025-12-16T03:10:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
196462025-12-16T03:10:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
196472025-12-16T03:10:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
196482025-12-16T03:10:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
196492025-12-16T03:10:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
196502025-12-16T03:10:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
196512025-12-16T03:10:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
196522025-12-16T03:10:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
196532025-12-16T03:10:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
196542025-12-16T03:10:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
196552025-12-16T03:10:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
196562025-12-16T03:10:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
196572025-12-16T03:10:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
196582025-12-16T03:10:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
196592025-12-16T03:10:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
196602025-12-16T03:10:09.319ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
196612025-12-16T03:10:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
196622025-12-16T03:10:09.321ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196632025-12-16T03:10:09.321ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196642025-12-16T03:10:09.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196652025-12-16T03:10:09.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196662025-12-16T03:10:09.321ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196672025-12-16T03:10:09.321ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196682025-12-16T03:10:09.321ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196692025-12-16T03:10:09.322ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
196702025-12-16T03:10:09.322ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
196712025-12-16T03:10:09.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
196722025-12-16T03:10:09.323ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
196732025-12-16T03:10:09.323ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
196742025-12-16T03:10:09.324ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
196752025-12-16T03:10:09.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
196762025-12-16T03:10:09.324ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196772025-12-16T03:10:09.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
196782025-12-16T03:10:09.325ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
196792025-12-16T03:10:09.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
196802025-12-16T03:10:09.327ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196812025-12-16T03:10:09.328ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
196822025-12-16T03:10:09.328ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
196832025-12-16T03:10:09.328ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
196842025-12-16T03:10:09.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
196852025-12-16T03:10:09.328ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
196862025-12-16T03:10:09.328ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
196872025-12-16T03:10:09.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
196882025-12-16T03:10:09.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
196892025-12-16T03:10:09.328ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
196902025-12-16T03:10:09.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
196912025-12-16T03:10:09.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
196922025-12-16T03:10:09.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196932025-12-16T03:10:09.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
196942025-12-16T03:10:09.328ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
196952025-12-16T03:10:09.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
196962025-12-16T03:10:09.329ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 10.25855ms iteration = 23 status = {"errors":[],"requests_created_ok":[]}
196972025-12-16T03:10:09.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
196982025-12-16T03:10:09.329ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
196992025-12-16T03:10:09.329ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 11.176432ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
197002025-12-16T03:10:09.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
197012025-12-16T03:10:09.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
197022025-12-16T03:10:09.330ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
197032025-12-16T03:10:09.331ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
197042025-12-16T03:10:09.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
197052025-12-16T03:10:09.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
197062025-12-16T03:10:09.331ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
197072025-12-16T03:10:09.331ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
197082025-12-16T03:10:09.331ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
197092025-12-16T03:10:09.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
197102025-12-16T03:10:09.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
197112025-12-16T03:10:09.332ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197122025-12-16T03:10:09.333ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
197132025-12-16T03:10:09.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
197142025-12-16T03:10:09.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
197152025-12-16T03:10:09.334ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 16.955775ms iteration = 45 status = "NoSitrep"
197162025-12-16T03:10:09.335ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
197172025-12-16T03:10:09.339ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
197182025-12-16T03:10:09.339ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
197192025-12-16T03:10:09.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
197202025-12-16T03:10:09.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
197212025-12-16T03:10:09.341ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
197222025-12-16T03:10:09.342ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
197232025-12-16T03:10:09.342ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
197242025-12-16T03:10:09.342ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 122.987758ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
197252025-12-16T03:10:09.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
197262025-12-16T03:10:09.342ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1616, tv_nsec: 950928548 } })
197272025-12-16T03:10:09.343ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
197282025-12-16T03:10:09.344ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1616, tv_nsec: 952079413 } })
197292025-12-16T03:10:09.344ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197302025-12-16T03:10:09.345ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
197312025-12-16T03:10:09.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
197322025-12-16T03:10:09.347ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
197332025-12-16T03:10:09.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
197342025-12-16T03:10:09.348ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = a51e37d5-b58a-4ed9-b46c-46ad89c1f4ad original_time_started = 2025-12-16 03:09:11.258385 UTC
197352025-12-16T03:10:09.348ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 30.8213ms iteration = 58 status = {"Loaded":{"collection_id":"a51e37d5-b58a-4ed9-b46c-46ad89c1f4ad","time_loaded":"2025-12-16T03:10:09.317786198Z","time_started":"2025-12-16T03:09:11.258385Z"}}
197362025-12-16T03:10:09.349ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
197372025-12-16T03:10:09.349ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
197382025-12-16T03:10:09.349ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
197392025-12-16T03:10:09.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
197402025-12-16T03:10:09.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
197412025-12-16T03:10:09.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
197422025-12-16T03:10:09.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
197432025-12-16T03:10:09.351ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 33.10507ms iteration = 23 status = 0
197442025-12-16T03:10:09.353ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
197452025-12-16T03:10:09.353ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
197462025-12-16T03:10:09.353ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 34.876232ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
197472025-12-16T03:10:09.356ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 180.157912ms iteration = 23 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
197482025-12-16T03:10:09.357ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
197492025-12-16T03:10:09.357ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
197502025-12-16T03:10:09.357ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 138.517685ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
197512025-12-16T03:10:09.358ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197522025-12-16T03:10:09.362ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
197532025-12-16T03:10:09.362ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
197542025-12-16T03:10:09.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
197552025-12-16T03:10:09.363ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197562025-12-16T03:10:09.366ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
197572025-12-16T03:10:09.368ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
197582025-12-16T03:10:09.368ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 49.993473ms iteration = 45 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
197592025-12-16T03:10:09.369ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197602025-12-16T03:10:09.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197612025-12-16T03:10:09.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197622025-12-16T03:10:09.374ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
197632025-12-16T03:10:09.374ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
197642025-12-16T03:10:09.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
197652025-12-16T03:10:09.375ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
197662025-12-16T03:10:09.376ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
197672025-12-16T03:10:09.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
197682025-12-16T03:10:09.377ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197692025-12-16T03:10:09.378ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197702025-12-16T03:10:09.382ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
197712025-12-16T03:10:09.382ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
197722025-12-16T03:10:09.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
197732025-12-16T03:10:09.382ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
197742025-12-16T03:10:09.382ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
197752025-12-16T03:10:09.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
197762025-12-16T03:10:09.384ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
197772025-12-16T03:10:09.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
197782025-12-16T03:10:09.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
197792025-12-16T03:10:09.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
197802025-12-16T03:10:09.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
197812025-12-16T03:10:09.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
197822025-12-16T03:10:09.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
197832025-12-16T03:10:09.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197842025-12-16T03:10:09.396ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
197852025-12-16T03:10:09.396ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
197862025-12-16T03:10:09.396ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
197872025-12-16T03:10:09.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
197882025-12-16T03:10:09.399ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197892025-12-16T03:10:09.400ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 81.550763ms iteration = 37 status = {"physical_disks_added":0}
197902025-12-16T03:10:09.401ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
197912025-12-16T03:10:09.401ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
197922025-12-16T03:10:09.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
197932025-12-16T03:10:09.404ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
197942025-12-16T03:10:09.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
197952025-12-16T03:10:09.406ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
197962025-12-16T03:10:09.406ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
197972025-12-16T03:10:09.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
197982025-12-16T03:10:09.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197992025-12-16T03:10:09.411ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
198002025-12-16T03:10:09.411ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
198012025-12-16T03:10:09.411ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-12-16 02:59:16.791641 UTC dns_group = internal generation = 1
198022025-12-16T03:10:09.411ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 234.472721ms iteration = 12 status = {"generation":1}
198032025-12-16T03:10:09.412ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
198042025-12-16T03:10:09.412ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 93.648557ms iteration = 67 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
198052025-12-16T03:10:09.412ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
198062025-12-16T03:10:09.413ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
198072025-12-16T03:10:09.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
198082025-12-16T03:10:09.414ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
198092025-12-16T03:10:09.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
198102025-12-16T03:10:09.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198112025-12-16T03:10:09.416ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198122025-12-16T03:10:09.416ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
198132025-12-16T03:10:09.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
198142025-12-16T03:10:09.419ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
198152025-12-16T03:10:09.420ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
198162025-12-16T03:10:09.420ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
198172025-12-16T03:10:09.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
198182025-12-16T03:10:09.422ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
198192025-12-16T03:10:09.422ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
198202025-12-16T03:10:09.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
198212025-12-16T03:10:09.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
198222025-12-16T03:10:09.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
198232025-12-16T03:10:09.426ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 108.10794ms iteration = 23 status = {"errors":[],"finish_invoked_ok":[]}
198242025-12-16T03:10:09.427ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
198252025-12-16T03:10:09.427ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 108.965171ms iteration = 133 status = {"config_updated":false}
198262025-12-16T03:10:09.428ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 109.266984ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
198272025-12-16T03:10:09.428ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
198282025-12-16T03:10:09.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
198292025-12-16T03:10:09.430ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198302025-12-16T03:10:09.433ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
198312025-12-16T03:10:09.433ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
198322025-12-16T03:10:09.433ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
198332025-12-16T03:10:09.434ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 115.360222ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
198342025-12-16T03:10:09.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
198352025-12-16T03:10:09.435ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
198362025-12-16T03:10:09.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
198372025-12-16T03:10:09.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
198382025-12-16T03:10:09.438ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
198392025-12-16T03:10:09.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
198402025-12-16T03:10:09.439ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
198412025-12-16T03:10:09.439ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
198422025-12-16T03:10:09.439ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
198432025-12-16T03:10:09.439ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
198442025-12-16T03:10:09.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
198452025-12-16T03:10:09.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
198462025-12-16T03:10:09.441ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198472025-12-16T03:10:09.445ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
198482025-12-16T03:10:09.445ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
198492025-12-16T03:10:09.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
198502025-12-16T03:10:09.447ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 128.574948ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
198512025-12-16T03:10:09.450ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
198522025-12-16T03:10:09.450ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
198532025-12-16T03:10:09.450ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "38eb7ac1-d1ff-4b5e-a26a-160886abbe07", "content-length": "26", "date": "Tue, 16 Dec 2025 03:10:09 GMT"} })
198542025-12-16T03:10:09.450ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
198552025-12-16T03:10:09.454ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
198562025-12-16T03:10:09.454ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 135.499866ms iteration = 34 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
198572025-12-16T03:10:09.454ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
198582025-12-16T03:10:09.455ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1617, tv_nsec: 63074108 } })
198592025-12-16T03:10:09.468ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
198602025-12-16T03:10:09.477ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2b346da2-5029-4762-b024-fccd46d69b68 step component = DeployNexusRecords step id = Ensure
198612025-12-16T03:10:09.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
198622025-12-16T03:10:09.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
198632025-12-16T03:10:09.478ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
198642025-12-16T03:10:09.478ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
198652025-12-16T03:10:09.479ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d3ce83e-fa1a-460c-ae33-8b68e62a8914", "content-length": "26", "date": "Tue, 16 Dec 2025 03:10:09 GMT"} })
198662025-12-16T03:10:09.479ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
198672025-12-16T03:10:09.480ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198682025-12-16T03:10:09.481ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
198692025-12-16T03:10:09.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
198702025-12-16T03:10:09.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
198712025-12-16T03:10:09.482ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
198722025-12-16T03:10:09.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
198732025-12-16T03:10:09.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
198742025-12-16T03:10:09.486ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
198752025-12-16T03:10:09.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
198762025-12-16T03:10:09.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198772025-12-16T03:10:09.492ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
198782025-12-16T03:10:09.501ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 182.882016ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
198792025-12-16T03:10:09.510ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
198802025-12-16T03:10:09.510ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
198812025-12-16T03:10:09.510ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
198822025-12-16T03:10:09.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
198832025-12-16T03:10:09.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198842025-12-16T03:10:09.515ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198852025-12-16T03:10:09.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
198862025-12-16T03:10:09.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
198872025-12-16T03:10:09.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
198882025-12-16T03:10:09.520ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
198892025-12-16T03:10:09.520ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
198902025-12-16T03:10:09.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
198912025-12-16T03:10:09.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
198922025-12-16T03:10:09.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
198932025-12-16T03:10:09.525ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
198942025-12-16T03:10:09.525ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
198952025-12-16T03:10:09.525ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-12-16 02:59:18.154440 UTC dns_group = external generation = 2
198962025-12-16T03:10:09.525ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 348.641037ms iteration = 12 status = {"generation":2}
198972025-12-16T03:10:09.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198982025-12-16T03:10:09.533ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
198992025-12-16T03:10:09.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199002025-12-16T03:10:09.533ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199012025-12-16T03:10:09.544ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
199022025-12-16T03:10:09.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
199032025-12-16T03:10:09.545ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2b346da2-5029-4762-b024-fccd46d69b68 step component = SledAgent step id = Ensure
199042025-12-16T03:10:09.545ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-12-16T02:59:05.020237Z, time_modified: 2025-12-16T02:59:05.020237Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }) }
199052025-12-16T03:10:09.545ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 369.091016ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]}},"default_endpoint":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
199062025-12-16T03:10:09.546ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199072025-12-16T03:10:09.550ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
199082025-12-16T03:10:09.550ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
199092025-12-16T03:10:09.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
199102025-12-16T03:10:09.559ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199112025-12-16T03:10:09.559ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199122025-12-16T03:10:09.562ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199132025-12-16T03:10:09.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199142025-12-16T03:10:09.565ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199152025-12-16T03:10:09.569ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
199162025-12-16T03:10:09.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
199172025-12-16T03:10:09.570ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
199182025-12-16T03:10:09.570ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199192025-12-16T03:10:09.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199202025-12-16T03:10:09.571ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
199212025-12-16T03:10:09.571ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
199222025-12-16T03:10:09.571ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 252.846833ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
199232025-12-16T03:10:09.572ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199242025-12-16T03:10:09.580ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
199252025-12-16T03:10:09.580ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199262025-12-16T03:10:09.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199272025-12-16T03:10:09.585ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199282025-12-16T03:10:09.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199292025-12-16T03:10:09.588ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199302025-12-16T03:10:09.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199312025-12-16T03:10:09.591ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199322025-12-16T03:10:09.597ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
199332025-12-16T03:10:09.597ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199342025-12-16T03:10:09.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199352025-12-16T03:10:09.600ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199362025-12-16T03:10:09.604ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
199372025-12-16T03:10:09.604ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199382025-12-16T03:10:09.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199392025-12-16T03:10:09.608ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199402025-12-16T03:10:09.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199412025-12-16T03:10:09.609ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199422025-12-16T03:10:09.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199432025-12-16T03:10:09.611ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199442025-12-16T03:10:09.616ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
199452025-12-16T03:10:09.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199462025-12-16T03:10:09.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199472025-12-16T03:10:09.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199482025-12-16T03:10:09.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199492025-12-16T03:10:09.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199502025-12-16T03:10:09.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
199512025-12-16T03:10:09.654ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1617, tv_nsec: 262360334 } })
199522025-12-16T03:10:09.659ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
199532025-12-16T03:10:09.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
199542025-12-16T03:10:09.661ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 484.945504ms iteration = 23 status = {}
199552025-12-16T03:10:09.662ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199562025-12-16T03:10:09.665ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
199572025-12-16T03:10:09.665ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
199582025-12-16T03:10:09.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
199592025-12-16T03:10:09.668ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199602025-12-16T03:10:09.671ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
199612025-12-16T03:10:09.671ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
199622025-12-16T03:10:09.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
199632025-12-16T03:10:09.703ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199642025-12-16T03:10:09.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199652025-12-16T03:10:09.706ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199662025-12-16T03:10:09.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199672025-12-16T03:10:09.709ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199682025-12-16T03:10:09.715ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
199692025-12-16T03:10:09.715ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199702025-12-16T03:10:09.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199712025-12-16T03:10:09.719ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199722025-12-16T03:10:09.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199732025-12-16T03:10:09.720ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199742025-12-16T03:10:09.724ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
199752025-12-16T03:10:09.724ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199762025-12-16T03:10:09.724ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199772025-12-16T03:10:09.727ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199782025-12-16T03:10:09.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199792025-12-16T03:10:09.729ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199802025-12-16T03:10:09.732ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
199812025-12-16T03:10:09.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199822025-12-16T03:10:09.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199832025-12-16T03:10:09.733ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
199842025-12-16T03:10:09.733ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 514.536739ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
199852025-12-16T03:10:09.735ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
199862025-12-16T03:10:09.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199872025-12-16T03:10:09.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199882025-12-16T03:10:09.736ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "90b6c1ca-e60e-4b46-a0c3-3783f28e03f3", "date": "Tue, 16 Dec 2025 03:10:09 GMT"} })
199892025-12-16T03:10:09.736ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
199902025-12-16T03:10:09.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199912025-12-16T03:10:09.738ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
199922025-12-16T03:10:09.738ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 502.572887ms iteration = 12 status = {"errors":[],"probes_by_sled":{"b1c76b61-acee-4be6-b582-973c938d6b6e":0}}
199932025-12-16T03:10:09.740ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
199942025-12-16T03:10:09.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199952025-12-16T03:10:09.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199962025-12-16T03:10:09.752ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
199972025-12-16T03:10:09.791ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
199982025-12-16T03:10:09.791ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
199992025-12-16T03:10:09.791ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = fdadd28d-6ec4-4d96-9790-ec1ae90b7e2f parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
200002025-12-16T03:10:09.791ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
200012025-12-16T03:10:09.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
200022025-12-16T03:10:09.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
200032025-12-16T03:10:09.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
200042025-12-16T03:10:09.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
200052025-12-16T03:10:09.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
200062025-12-16T03:10:09.801ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
200072025-12-16T03:10:09.801ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
200082025-12-16T03:10:09.801ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 583.918958ms iteration = 28 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
200092025-12-16T03:10:09.809ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
200102025-12-16T03:10:09.810ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "fbd39b47-3f55-4a3e-afd3-2eb40316f2eb", "date": "Tue, 16 Dec 2025 03:10:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
200112025-12-16T03:10:09.810ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
200122025-12-16T03:10:09.812ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "8cf445c1-32c2-4236-9db8-4999089c2dba", "date": "Tue, 16 Dec 2025 03:10:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
200132025-12-16T03:10:09.812ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
200142025-12-16T03:10:09.815ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
200152025-12-16T03:10:09.815ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 638.386037ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
200162025-12-16T03:10:09.841ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
200172025-12-16T03:10:09.844ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "1db4f920-a966-4d5c-81ff-95389e1a9b11", "date": "Tue, 16 Dec 2025 03:10:09 GMT"} }) sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
200182025-12-16T03:10:09.845ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1617, tv_nsec: 453648387 } })
200192025-12-16T03:10:09.847ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2b346da2-5029-4762-b024-fccd46d69b68 step component = FirewallRules step id = Ensure
200202025-12-16T03:10:09.847ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200212025-12-16T03:10:09.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200222025-12-16T03:10:09.850ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200232025-12-16T03:10:09.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200242025-12-16T03:10:09.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200252025-12-16T03:10:09.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200262025-12-16T03:10:09.855ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200272025-12-16T03:10:09.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200282025-12-16T03:10:09.858ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200292025-12-16T03:10:09.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200302025-12-16T03:10:09.859ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200312025-12-16T03:10:09.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200322025-12-16T03:10:09.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
200332025-12-16T03:10:09.866ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
200342025-12-16T03:10:09.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200352025-12-16T03:10:09.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200362025-12-16T03:10:09.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200372025-12-16T03:10:09.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200382025-12-16T03:10:09.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200392025-12-16T03:10:09.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200402025-12-16T03:10:09.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
200412025-12-16T03:10:09.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
200422025-12-16T03:10:09.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200432025-12-16T03:10:09.882ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
200442025-12-16T03:10:09.882ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
200452025-12-16T03:10:09.882ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
200462025-12-16T03:10:09.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200472025-12-16T03:10:09.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200482025-12-16T03:10:09.886ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "bbdf4a8c-a740-4372-bc7f-206f06da121e", "content-length": "372", "date": "Tue, 16 Dec 2025 03:10:09 GMT"} })
200492025-12-16T03:10:09.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200502025-12-16T03:10:09.886ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
200512025-12-16T03:10:09.886ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "642533a0-98e8-48d4-af8e-8ef97c7de156", "content-length": "691", "date": "Tue, 16 Dec 2025 03:10:09 GMT"} })
200522025-12-16T03:10:09.886ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
200532025-12-16T03:10:09.886ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
200542025-12-16T03:10:09.886ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
200552025-12-16T03:10:09.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
200562025-12-16T03:10:09.888ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 569.400774ms iteration = 23 status = {}
200572025-12-16T03:10:09.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200582025-12-16T03:10:09.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200592025-12-16T03:10:09.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200602025-12-16T03:10:09.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200612025-12-16T03:10:09.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200622025-12-16T03:10:09.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200632025-12-16T03:10:09.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200642025-12-16T03:10:09.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200652025-12-16T03:10:09.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200662025-12-16T03:10:09.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200672025-12-16T03:10:09.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
200682025-12-16T03:10:09.903ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
200692025-12-16T03:10:09.903ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
200702025-12-16T03:10:09.903ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
200712025-12-16T03:10:09.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200722025-12-16T03:10:09.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200732025-12-16T03:10:09.905ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200742025-12-16T03:10:09.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200752025-12-16T03:10:09.908ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200762025-12-16T03:10:09.908ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "4d8643df-28d9-401d-93cb-c369c29d74d6", "date": "Tue, 16 Dec 2025 03:10:09 GMT"} })
200772025-12-16T03:10:09.908ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
200782025-12-16T03:10:09.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200792025-12-16T03:10:09.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
200802025-12-16T03:10:09.911ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
200812025-12-16T03:10:09.911ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200822025-12-16T03:10:09.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200832025-12-16T03:10:09.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
200842025-12-16T03:10:09.913ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200852025-12-16T03:10:09.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200862025-12-16T03:10:09.914ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
200872025-12-16T03:10:09.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
200882025-12-16T03:10:09.915ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200892025-12-16T03:10:09.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200902025-12-16T03:10:09.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200912025-12-16T03:10:09.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200922025-12-16T03:10:09.918ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
200932025-12-16T03:10:09.919ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
200942025-12-16T03:10:09.919ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
200952025-12-16T03:10:09.923ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
200962025-12-16T03:10:09.923ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200972025-12-16T03:10:09.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200982025-12-16T03:10:09.925ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200992025-12-16T03:10:09.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
201002025-12-16T03:10:09.925ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "671c992c-669a-4243-9d56-9969a0fd0d23", "date": "Tue, 16 Dec 2025 03:10:09 GMT"} })
201012025-12-16T03:10:09.927ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
201022025-12-16T03:10:09.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
201032025-12-16T03:10:09.927ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 609.217548ms iteration = 34 status = {}
201042025-12-16T03:10:09.927ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 35 reason = Dependency
201052025-12-16T03:10:09.927ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
201062025-12-16T03:10:09.927ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
201072025-12-16T03:10:09.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
201082025-12-16T03:10:09.929ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201092025-12-16T03:10:09.929ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201102025-12-16T03:10:09.932ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
201112025-12-16T03:10:09.933ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
201122025-12-16T03:10:09.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
201132025-12-16T03:10:09.934ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
201142025-12-16T03:10:09.934ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
201152025-12-16T03:10:09.934ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201162025-12-16T03:10:09.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
201172025-12-16T03:10:09.936ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
201182025-12-16T03:10:09.936ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
201192025-12-16T03:10:09.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
201202025-12-16T03:10:09.938ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
201212025-12-16T03:10:09.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
201222025-12-16T03:10:09.940ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
201232025-12-16T03:10:09.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
201242025-12-16T03:10:09.942ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
201252025-12-16T03:10:09.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
201262025-12-16T03:10:09.944ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
201272025-12-16T03:10:09.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
201282025-12-16T03:10:09.946ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
201292025-12-16T03:10:09.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
201302025-12-16T03:10:09.948ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
201312025-12-16T03:10:09.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
201322025-12-16T03:10:09.949ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
201332025-12-16T03:10:09.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
201342025-12-16T03:10:09.951ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201352025-12-16T03:10:09.956ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
201362025-12-16T03:10:09.956ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
201372025-12-16T03:10:09.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
201382025-12-16T03:10:09.957ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
201392025-12-16T03:10:09.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
201402025-12-16T03:10:09.958ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
201412025-12-16T03:10:09.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
201422025-12-16T03:10:09.960ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201432025-12-16T03:10:09.964ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
201442025-12-16T03:10:09.964ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
201452025-12-16T03:10:09.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
201462025-12-16T03:10:09.969ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
201472025-12-16T03:10:09.969ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "e3c89c40-cb33-4b0e-b2dd-f7edec27215c", "date": "Tue, 16 Dec 2025 03:10:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
201482025-12-16T03:10:09.969ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
201492025-12-16T03:10:09.971ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }], Name("external-dns"): [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
201502025-12-16T03:10:09.971ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
201512025-12-16T03:10:09.971ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "1e94a0a1-530a-4ad6-b531-80a291214210", "date": "Tue, 16 Dec 2025 03:10:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
201522025-12-16T03:10:09.971ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
201532025-12-16T03:10:09.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
201542025-12-16T03:10:09.972ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201552025-12-16T03:10:09.974ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "210889e3-c0e4-4345-b32c-2dac52fa1b9c", "date": "Tue, 16 Dec 2025 03:10:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
201562025-12-16T03:10:09.974ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
201572025-12-16T03:10:09.975ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
201582025-12-16T03:10:09.975ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
201592025-12-16T03:10:09.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
201602025-12-16T03:10:09.976ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
201612025-12-16T03:10:09.976ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 799.668982ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
201622025-12-16T03:10:09.976ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
201632025-12-16T03:10:09.976ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
201642025-12-16T03:10:09.976ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
201652025-12-16T03:10:09.979ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
201662025-12-16T03:10:09.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
201672025-12-16T03:10:09.981ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
201682025-12-16T03:10:09.981ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
201692025-12-16T03:10:09.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
201702025-12-16T03:10:09.993ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
201712025-12-16T03:10:09.993ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
201722025-12-16T03:10:09.994ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
201732025-12-16T03:10:09.994ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
201742025-12-16T03:10:09.994ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e11ef76-105e-49ee-a06b-5205793d3d46", "content-length": "372", "date": "Tue, 16 Dec 2025 03:10:09 GMT"} })
201752025-12-16T03:10:09.994ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
201762025-12-16T03:10:09.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
201772025-12-16T03:10:09.994ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
201782025-12-16T03:10:09.994ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
201792025-12-16T03:10:09.994ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
201802025-12-16T03:10:09.994ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
201812025-12-16T03:10:09.994ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
201822025-12-16T03:10:09.994ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
201832025-12-16T03:10:09.995ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
201842025-12-16T03:10:09.995ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
201852025-12-16T03:10:09.995ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
201862025-12-16T03:10:09.995ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
201872025-12-16T03:10:09.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
201882025-12-16T03:10:09.998ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
201892025-12-16T03:10:09.998ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
201902025-12-16T03:10:10.008ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
201912025-12-16T03:10:10.008ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
201922025-12-16T03:10:10.010ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "6c2379b2-ad5c-4034-9dce-13260b4c03fc", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
201932025-12-16T03:10:10.010ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
201942025-12-16T03:10:10.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
201952025-12-16T03:10:10.012ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
201962025-12-16T03:10:10.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
201972025-12-16T03:10:10.016ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
201982025-12-16T03:10:10.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
201992025-12-16T03:10:10.019ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
202002025-12-16T03:10:10.019ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
202012025-12-16T03:10:10.024ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "c613a369-7d35-43cb-9ab9-1a4f04578595", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
202022025-12-16T03:10:10.026ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 98.616549ms iteration = 35 status = {}
202032025-12-16T03:10:10.047ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), time_created: 2025-12-16T02:59:15.195809Z, time_modified: 2025-12-16T02:59:15.195809Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
202042025-12-16T03:10:10.047ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
202052025-12-16T03:10:10.080ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202062025-12-16T03:10:10.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202072025-12-16T03:10:10.084ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202082025-12-16T03:10:10.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202092025-12-16T03:10:10.086ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202102025-12-16T03:10:10.090ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
202112025-12-16T03:10:10.090ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
202122025-12-16T03:10:10.094ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "f039b57a-9039-4aff-aa37-e8e88da5ed88", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
202132025-12-16T03:10:10.096ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
202142025-12-16T03:10:10.096ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2b346da2-5029-4762-b024-fccd46d69b68 step component = Dns step id = Ensure
202152025-12-16T03:10:10.096ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
202162025-12-16T03:10:10.096ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202172025-12-16T03:10:10.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202182025-12-16T03:10:10.097ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202192025-12-16T03:10:10.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202202025-12-16T03:10:10.099ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202212025-12-16T03:10:10.103ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
202222025-12-16T03:10:10.104ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
202232025-12-16T03:10:10.104ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
202242025-12-16T03:10:10.104ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
202252025-12-16T03:10:10.104ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202262025-12-16T03:10:10.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202272025-12-16T03:10:10.107ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202282025-12-16T03:10:10.111ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
202292025-12-16T03:10:10.111ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202302025-12-16T03:10:10.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202312025-12-16T03:10:10.118ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "3287244c-9374-49e4-b1d4-f1b847976e20", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
202322025-12-16T03:10:10.118ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
202332025-12-16T03:10:10.118ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
202342025-12-16T03:10:10.119ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "9cffc392-bd7d-4a0e-9708-8d8ecaf5101b", "content-length": "682", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
202352025-12-16T03:10:10.119ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
202362025-12-16T03:10:10.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
202372025-12-16T03:10:10.122ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
202382025-12-16T03:10:10.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
202392025-12-16T03:10:10.127ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
202402025-12-16T03:10:10.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
202412025-12-16T03:10:10.129ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
202422025-12-16T03:10:10.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
202432025-12-16T03:10:10.137ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 819.653924ms iteration = 23 status = {}
202442025-12-16T03:10:10.152ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
202452025-12-16T03:10:10.152ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
202462025-12-16T03:10:10.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202472025-12-16T03:10:10.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202482025-12-16T03:10:10.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202492025-12-16T03:10:10.157ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
202502025-12-16T03:10:10.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202512025-12-16T03:10:10.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202522025-12-16T03:10:10.160ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
202532025-12-16T03:10:10.160ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
202542025-12-16T03:10:10.160ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
202552025-12-16T03:10:10.160ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202562025-12-16T03:10:10.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202572025-12-16T03:10:10.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202582025-12-16T03:10:10.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202592025-12-16T03:10:10.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202602025-12-16T03:10:10.166ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
202612025-12-16T03:10:10.167ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
202622025-12-16T03:10:10.167ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
202632025-12-16T03:10:10.167ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
202642025-12-16T03:10:10.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202652025-12-16T03:10:10.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202662025-12-16T03:10:10.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202672025-12-16T03:10:10.172ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
202682025-12-16T03:10:10.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202692025-12-16T03:10:10.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202702025-12-16T03:10:10.174ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
202712025-12-16T03:10:10.174ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
202722025-12-16T03:10:10.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202732025-12-16T03:10:10.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202742025-12-16T03:10:10.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202752025-12-16T03:10:10.179ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
202762025-12-16T03:10:10.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202772025-12-16T03:10:10.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202782025-12-16T03:10:10.181ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
202792025-12-16T03:10:10.181ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
202802025-12-16T03:10:10.181ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202812025-12-16T03:10:10.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202822025-12-16T03:10:10.183ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202832025-12-16T03:10:10.185ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
202842025-12-16T03:10:10.185ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202852025-12-16T03:10:10.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202862025-12-16T03:10:10.187ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202872025-12-16T03:10:10.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202882025-12-16T03:10:10.188ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202892025-12-16T03:10:10.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202902025-12-16T03:10:10.190ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202912025-12-16T03:10:10.193ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
202922025-12-16T03:10:10.195ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
202932025-12-16T03:10:10.195ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
202942025-12-16T03:10:10.195ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2b346da2-5029-4762-b024-fccd46d69b68 step component = OmicronZones step id = Cleanup
202952025-12-16T03:10:10.195ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2b346da2-5029-4762-b024-fccd46d69b68 step component = OmicronZones step id = Cleanup
202962025-12-16T03:10:10.195ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2b346da2-5029-4762-b024-fccd46d69b68 step component = PhysicalDisks step id = Cleanup
202972025-12-16T03:10:10.195ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2b346da2-5029-4762-b024-fccd46d69b68 step component = Clickhouse step id = Ensure
202982025-12-16T03:10:10.195ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2b346da2-5029-4762-b024-fccd46d69b68 step component = Clickhouse step id = Ensure
202992025-12-16T03:10:10.225ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
203002025-12-16T03:10:10.230ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "f8346d5b-5a2a-429b-9aa9-94f6378259dc", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
203012025-12-16T03:10:10.233ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2b346da2-5029-4762-b024-fccd46d69b68 step component = SupportBundles step id = Cleanup
203022025-12-16T03:10:10.233ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203032025-12-16T03:10:10.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203042025-12-16T03:10:10.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203052025-12-16T03:10:10.236ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
203062025-12-16T03:10:10.236ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203072025-12-16T03:10:10.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203082025-12-16T03:10:10.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203092025-12-16T03:10:10.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203102025-12-16T03:10:10.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203112025-12-16T03:10:10.243ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
203122025-12-16T03:10:10.271ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 4
203132025-12-16T03:10:10.271ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221
203142025-12-16T03:10:10.291ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2b346da2-5029-4762-b024-fccd46d69b68 step component = OmicronZones step id = Cleanup
203152025-12-16T03:10:10.291ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
203162025-12-16T03:10:10.291ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
203172025-12-16T03:10:10.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203182025-12-16T03:10:10.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203192025-12-16T03:10:10.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203202025-12-16T03:10:10.294ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
203212025-12-16T03:10:10.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203222025-12-16T03:10:10.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203232025-12-16T03:10:10.302ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
203242025-12-16T03:10:10.302ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:49209
203252025-12-16T03:10:10.302ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:49209 req_id = 96e8b2a0-98d0-4494-bdee-61292b2d796a uri = /metrics/producers version = 1.0.0
203262025-12-16T03:10:10.302ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:49209 req_id = 96e8b2a0-98d0-4494-bdee-61292b2d796a roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
203272025-12-16T03:10:10.303ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:49209 req_id = 96e8b2a0-98d0-4494-bdee-61292b2d796a resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
203282025-12-16T03:10:10.335ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
203292025-12-16T03:10:10.335ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(a91734db-9ac7-4793-9c32-ddbb21f8d986 (blueprint), false)
203302025-12-16T03:10:10.335ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2b346da2-5029-4762-b024-fccd46d69b68 step component = Cockroach step id = Ensure
203312025-12-16T03:10:10.335ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203322025-12-16T03:10:10.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203332025-12-16T03:10:10.356ZINFOnexus: assigned collector to new producer collector_id = e5b81e4f-90b9-43ba-ba56-584fd121e601 file = nexus/src/app/oximeter.rs:76 producer_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
203342025-12-16T03:10:10.356ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 54340 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:49209 req_id = 96e8b2a0-98d0-4494-bdee-61292b2d796a response_code = 201 uri = /metrics/producers
203352025-12-16T03:10:10.357ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "96e8b2a0-98d0-4494-bdee-61292b2d796a", "content-length": "41", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
203362025-12-16T03:10:10.358ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
203372025-12-16T03:10:10.363ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
203382025-12-16T03:10:10.363ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2b346da2-5029-4762-b024-fccd46d69b68 step component = MgsUpdates step id = Ensure
203392025-12-16T03:10:10.363ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 14 reason = Dependency
203402025-12-16T03:10:10.363ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 35 reason = Dependency
203412025-12-16T03:10:10.363ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
203422025-12-16T03:10:10.363ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
203432025-12-16T03:10:10.364ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.187363664s iteration = 13 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"3adfcc11-1048-42b1-b3a7-e0a64888df5a","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"3adfcc11-1048-42b1-b3a7-e0a64888df5a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3269,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":103798965,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":103798925,"secs":0}},"event_index":1,"execution_id":"3adfcc11-1048-42b1-b3a7-e0a64888df5a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":103812515,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":196588970,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":196588930,"secs":0}},"event_index":2,"execution_id":"3adfcc11-1048-42b1-b3a7-e0a64888df5a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":300424615,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":67844050,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":67844010,"secs":0}},"event_index":3,"execution_id":"3adfcc11-1048-42b1-b3a7-e0a64888df5a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":368291855,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":301799774,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":301799734,"secs":0}},"event_index":4,"execution_id":"3adfcc11-1048-42b1-b3a7-e0a64888df5a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":670107509,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":249150697,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":249150657,"secs":0}},"event_index":5,"execution_id":"3adfcc11-1048-42b1-b3a7-e0a64888df5a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":919276807,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":98956703,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":98956663,"secs":0}},"event_index":6,"execution_id":"3adfcc11-1048-42b1-b3a7-e0a64888df5a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":18251110,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4140,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4100,"secs":0}},"event_index":7,"execution_id":"3adfcc11-1048-42b1-b3a7-e0a64888df5a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":18270411,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2800,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2760,"secs":0}},"event_index":8,"execution_id":"3adfcc11-1048-42b1-b3a7-e0a64888df5a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":18275311,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2930,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":2890,"secs":0}},"event_index":9,"execution_id":"3adfcc11-1048-42b1-b3a7-e0a64888df5a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":18280151,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1660,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1620,"secs":0}},"event_index":10,"execution_id":"3adfcc11-1048-42b1-b3a7-e0a64888df5a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":18283621,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":37918640,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":37918600,"secs":0}},"event_index":11,"execution_id":"3adfcc11-1048-42b1-b3a7-e0a64888df5a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":56203811,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":58285889,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":58285849,"secs":0}},"event_index":12,"execution_id":"3adfcc11-1048-42b1-b3a7-e0a64888df5a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":114502300,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":44536344,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":44536304,"secs":0}},"event_index":13,"execution_id":"3adfcc11-1048-42b1-b3a7-e0a64888df5a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":159054494,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":27923194,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":27923154,"secs":0}},"event_index":14,"execution_id":"3adfcc11-1048-42b1-b3a7-e0a64888df5a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":186988428,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5260,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5230,"secs":0}},"event_index":15,"execution_id":"3adfcc11-1048-42b1-b3a7-e0a64888df5a","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":187002939,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986"}
203442025-12-16T03:10:10.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
203452025-12-16T03:10:10.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
203462025-12-16T03:10:10.365ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203472025-12-16T03:10:10.366ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203482025-12-16T03:10:10.368ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
203492025-12-16T03:10:10.368ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
203502025-12-16T03:10:10.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
203512025-12-16T03:10:10.369ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
203522025-12-16T03:10:10.369ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
203532025-12-16T03:10:10.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
203542025-12-16T03:10:10.370ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
203552025-12-16T03:10:10.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
203562025-12-16T03:10:10.372ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203572025-12-16T03:10:10.375ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
203582025-12-16T03:10:10.375ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
203592025-12-16T03:10:10.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
203602025-12-16T03:10:10.376ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
203612025-12-16T03:10:10.376ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 12.952204ms iteration = 35 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
203622025-12-16T03:10:10.443ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(71765244-afb4-4b31-85ba-c7a3812878cc, 0), (d415f2c0-1c8c-4f2c-aa1d-d5128edd7707, 0), (c98952ab-e784-47f5-b72e-5889b99827e8, 0), (a51e37d5-b58a-4ed9-b46c-46ad89c1f4ad, 0)] collection_id = 71765244-afb4-4b31-85ba-c7a3812878cc
203632025-12-16T03:10:10.443ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
203642025-12-16T03:10:10.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
203652025-12-16T03:10:10.609ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 71765244-afb4-4b31-85ba-c7a3812878cc file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
203662025-12-16T03:10:10.609ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
203672025-12-16T03:10:10.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
203682025-12-16T03:10:10.612ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(d415f2c0-1c8c-4f2c-aa1d-d5128edd7707, 0), (c98952ab-e784-47f5-b72e-5889b99827e8, 0), (a51e37d5-b58a-4ed9-b46c-46ad89c1f4ad, 0)]
203692025-12-16T03:10:10.613ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1618, tv_nsec: 221092891 } })
203702025-12-16T03:10:10.644ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("20488683-5778-4090-b36e-e6f0d80a682b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("35dbaad1-8536-44a0-8a0a-35d7c39dcfbb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b57fa226-2428-4a54-b85d-60409e28b484.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("d812a3e2-48ae-4812-a1cc-7eb7623e4e00.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::5)) }], valid_until: Instant { tv_sec: 1618, tv_nsec: 252913424 } })
203712025-12-16T03:10:10.645ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
203722025-12-16T03:10:10.645ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
203732025-12-16T03:10:10.645ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
203742025-12-16T03:10:10.646ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0643179-6d50-4749-aa49-9b577920a4c7", "content-length": "476", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
203752025-12-16T03:10:10.646ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
203762025-12-16T03:10:10.646ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "281be637-fee3-453c-b9a2-f0ff3604b153", "content-length": "734", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
203772025-12-16T03:10:10.646ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
203782025-12-16T03:10:10.647ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7629c2d6-e9f7-419e-b8de-310dd47cb57e", "content-length": "111", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
203792025-12-16T03:10:10.647ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
203802025-12-16T03:10:10.648ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "72f7aa5a-2519-43c5-a3db-37d1c33c39bc", "content-length": "111", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
203812025-12-16T03:10:10.648ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
203822025-12-16T03:10:10.648ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6626079c-27d8-4bb2-ba9f-65bb435d0467", "content-length": "163", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
203832025-12-16T03:10:10.648ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
203842025-12-16T03:10:10.649ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "950cc723-3028-4d4c-a697-69638a0a679c", "content-length": "163", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
203852025-12-16T03:10:10.649ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
203862025-12-16T03:10:10.649ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "41c23f56-eafb-4d1d-96eb-1b27edd34632", "content-length": "166", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
203872025-12-16T03:10:10.649ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
203882025-12-16T03:10:10.650ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b5c648d-c955-4419-aa62-78aa14d8cdc9", "content-length": "166", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
203892025-12-16T03:10:10.650ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
203902025-12-16T03:10:10.650ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e696723-54ab-4aa8-9eb7-a53ee62bd396", "content-length": "702", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
203912025-12-16T03:10:10.650ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
203922025-12-16T03:10:10.651ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "54666e7f-264d-477a-a80c-376c8152158b", "content-length": "727", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
203932025-12-16T03:10:10.651ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
203942025-12-16T03:10:10.651ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "47066b3f-dacb-407c-8b97-2309ef2f9093", "content-length": "729", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
203952025-12-16T03:10:10.651ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
203962025-12-16T03:10:10.651ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "44dd08df-7287-44ce-b768-62df1866fa06", "content-length": "728", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
203972025-12-16T03:10:10.651ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
203982025-12-16T03:10:10.651ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3b5805a-e34e-4f93-844b-6abac4fffe3e", "content-length": "722", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
203992025-12-16T03:10:10.651ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
204002025-12-16T03:10:10.652ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ca46f0f-5876-460e-9a6c-f0bc60840add", "content-length": "10", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204012025-12-16T03:10:10.652ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
204022025-12-16T03:10:10.652ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b54bb9e1-960a-4c04-8920-fdb45b46de78", "content-length": "144", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204032025-12-16T03:10:10.652ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
204042025-12-16T03:10:10.652ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c40d1f3-aac1-4fa2-a927-0a897dda6aec", "content-length": "147", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204052025-12-16T03:10:10.652ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
204062025-12-16T03:10:10.653ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7848918-4de6-487f-a727-d183593a68d4", "content-length": "109", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204072025-12-16T03:10:10.653ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
204082025-12-16T03:10:10.654ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "35e388c1-9e44-419d-91de-fcef5ff037c1", "content-length": "109", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204092025-12-16T03:10:10.654ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
204102025-12-16T03:10:10.654ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0a561e8-0a47-468f-beb9-98cd35508d0b", "content-length": "163", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204112025-12-16T03:10:10.654ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
204122025-12-16T03:10:10.655ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "87236b9c-3ebc-4ec7-aedf-27d521369c0f", "content-length": "163", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204132025-12-16T03:10:10.655ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
204142025-12-16T03:10:10.655ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d42d8861-2fd4-439b-be98-144f21fd4095", "content-length": "166", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204152025-12-16T03:10:10.655ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
204162025-12-16T03:10:10.656ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "326e3e86-6483-45b6-89d9-9e2e086f06d0", "content-length": "166", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204172025-12-16T03:10:10.656ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
204182025-12-16T03:10:10.656ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd6c7dac-92d9-4b97-9760-27d924fd0bb7", "content-length": "702", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204192025-12-16T03:10:10.656ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
204202025-12-16T03:10:10.657ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea2e369c-3b03-4785-997d-3e0a01b8b798", "content-length": "727", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204212025-12-16T03:10:10.657ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
204222025-12-16T03:10:10.657ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3592664e-6473-49cc-b112-88f5f6f4f9ae", "content-length": "729", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204232025-12-16T03:10:10.657ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
204242025-12-16T03:10:10.657ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1bf94e0-e4fd-49fa-85c7-a3e251e25a45", "content-length": "728", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204252025-12-16T03:10:10.657ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
204262025-12-16T03:10:10.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
204272025-12-16T03:10:10.661ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
204282025-12-16T03:10:10.664ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
204292025-12-16T03:10:10.664ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
204302025-12-16T03:10:10.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
204312025-12-16T03:10:10.666ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
204322025-12-16T03:10:10.668ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
204332025-12-16T03:10:10.668ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
204342025-12-16T03:10:10.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
204352025-12-16T03:10:10.703ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
204362025-12-16T03:10:10.703ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
204372025-12-16T03:10:10.772ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "5432a3f2-6240-43e6-86e0-90bdc010fa08", "content-length": "95612", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204382025-12-16T03:10:10.775ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
204392025-12-16T03:10:10.775ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
204402025-12-16T03:10:10.775ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
204412025-12-16T03:10:10.775ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
204422025-12-16T03:10:10.775ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
204432025-12-16T03:10:10.775ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
204442025-12-16T03:10:10.775ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
204452025-12-16T03:10:10.775ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
204462025-12-16T03:10:10.775ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5c3e2a8-3fb3-4538-821d-250df69cb94c", "content-length": "64", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204472025-12-16T03:10:10.775ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "74bf0fb1-f221-4047-b027-56a26c916741", "content-length": "64", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204482025-12-16T03:10:10.775ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
204492025-12-16T03:10:10.775ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8bd3df4a-252b-4805-898b-834e210f05b1", "content-length": "64", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204502025-12-16T03:10:10.775ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
204512025-12-16T03:10:10.775ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
204522025-12-16T03:10:10.775ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "cdb739cb-40ea-4a66-be65-43309a166328", "content-length": "64", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204532025-12-16T03:10:10.775ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
204542025-12-16T03:10:10.775ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b98fc6a2-7f12-4311-998c-2e979f524ba2", "content-length": "64", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204552025-12-16T03:10:10.775ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
204562025-12-16T03:10:10.814ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "f45fc269-bf5d-470c-b89e-1a3410caaef7", "content-length": "410816", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204572025-12-16T03:10:10.814ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "28d9075d-280f-4ce6-b100-f0fe6fb9e552", "content-length": "409370", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204582025-12-16T03:10:10.814ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "c44e6b4f-8e50-4e07-9d46-1b28a74f2fe5", "content-length": "438856", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204592025-12-16T03:10:10.814ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6ff2de0-4ff0-4f49-8adc-4a989022ce5d", "content-length": "411551", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204602025-12-16T03:10:10.816ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "f30c62e9-1589-47f4-a13f-7916c3917851", "content-length": "410554", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204612025-12-16T03:10:10.819ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
204622025-12-16T03:10:10.819ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
204632025-12-16T03:10:10.819ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
204642025-12-16T03:10:10.819ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
204652025-12-16T03:10:10.819ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
204662025-12-16T03:10:10.849ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c (omicron_zone)
204672025-12-16T03:10:10.849ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
204682025-12-16T03:10:10.853ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "53be33be-844a-4018-aaf7-f2e659a16ff5", "content-length": "124", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204692025-12-16T03:10:10.855ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
204702025-12-16T03:10:10.950ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
204712025-12-16T03:10:10.950ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
204722025-12-16T03:10:10.951ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "5067740e-2d49-4c2c-ba73-2d5a12044a1d", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204732025-12-16T03:10:10.951ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
204742025-12-16T03:10:10.952ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
204752025-12-16T03:10:10.952ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
204762025-12-16T03:10:10.953ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "de7ea2dd-4540-4f88-ab7c-3eac52f74ed8", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204772025-12-16T03:10:10.953ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
204782025-12-16T03:10:10.955ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
204792025-12-16T03:10:10.955ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
204802025-12-16T03:10:10.955ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "79628d8b-de93-493e-924d-42decf19ccdc", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:10:10 GMT"} })
204812025-12-16T03:10:10.955ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
204822025-12-16T03:10:10.956ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
204832025-12-16T03:10:10.956ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
204842025-12-16T03:10:10.958ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
204852025-12-16T03:10:10.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
204862025-12-16T03:10:10.960ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
204872025-12-16T03:10:10.962ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
204882025-12-16T03:10:10.962ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
204892025-12-16T03:10:10.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
204902025-12-16T03:10:11.069ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 296 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = c8d15569-4ac8-42fb-b2ac-c3a6b5b4c98b response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
204912025-12-16T03:10:11.165ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 4cf7dcb3-4693-4eaf-af85-9f40dc98c412 file = nexus/db-queries/src/db/datastore/inventory.rs:1613
204922025-12-16T03:10:11.165ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 4cf7dcb3-4693-4eaf-af85-9f40dc98c412 time_started = 2025-12-16 03:10:10.645891 UTC
204932025-12-16T03:10:11.165ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 40 reason = Dependency
204942025-12-16T03:10:11.165ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
204952025-12-16T03:10:11.165ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 59 reason = Dependency
204962025-12-16T03:10:11.165ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 20.05\u{b5}s iteration = 40 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
204972025-12-16T03:10:11.165ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
204982025-12-16T03:10:11.165ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 801.782588ms iteration = 14 status = {"collection_id":"4cf7dcb3-4693-4eaf-af85-9f40dc98c412","time_done":"2025-12-16 03:10:10.956851 UTC","time_started":"2025-12-16 03:10:10.645891 UTC"}
204992025-12-16T03:10:11.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
205002025-12-16T03:10:11.167ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205012025-12-16T03:10:11.171ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
205022025-12-16T03:10:11.171ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
205032025-12-16T03:10:11.171ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
205042025-12-16T03:10:11.173ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
205052025-12-16T03:10:11.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
205062025-12-16T03:10:11.223ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 58.242168ms iteration = 59 status = {"Loaded":{"collection_id":"4cf7dcb3-4693-4eaf-af85-9f40dc98c412","time_loaded":"2025-12-16T03:10:11.165710811Z","time_started":"2025-12-16T03:10:10.645891Z"}}
205072025-12-16T03:10:11.223ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 29 reason = Dependency
205082025-12-16T03:10:11.223ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 38 reason = Dependency
205092025-12-16T03:10:11.223ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 41 reason = Dependency
205102025-12-16T03:10:11.224ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
205112025-12-16T03:10:11.224ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
205122025-12-16T03:10:11.224ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
205132025-12-16T03:10:11.224ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 13.01\u{b5}s iteration = 41 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
205142025-12-16T03:10:11.224ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 18 reason = Dependency
205152025-12-16T03:10:11.224ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
205162025-12-16T03:10:11.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
205172025-12-16T03:10:11.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
205182025-12-16T03:10:11.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
205192025-12-16T03:10:11.225ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205202025-12-16T03:10:11.226ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205212025-12-16T03:10:11.226ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205222025-12-16T03:10:11.229ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
205232025-12-16T03:10:11.229ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
205242025-12-16T03:10:11.229ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
205252025-12-16T03:10:11.229ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
205262025-12-16T03:10:11.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
205272025-12-16T03:10:11.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
205282025-12-16T03:10:11.230ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
205292025-12-16T03:10:11.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
205302025-12-16T03:10:11.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
205312025-12-16T03:10:11.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205322025-12-16T03:10:11.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205332025-12-16T03:10:11.231ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
205342025-12-16T03:10:11.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
205352025-12-16T03:10:11.233ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205362025-12-16T03:10:11.234ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
205372025-12-16T03:10:11.234ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
205382025-12-16T03:10:11.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
205392025-12-16T03:10:11.234ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
205402025-12-16T03:10:11.234ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
205412025-12-16T03:10:11.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
205422025-12-16T03:10:11.236ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
205432025-12-16T03:10:11.236ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
205442025-12-16T03:10:11.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
205452025-12-16T03:10:11.236ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
205462025-12-16T03:10:11.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
205472025-12-16T03:10:11.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
205482025-12-16T03:10:11.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
205492025-12-16T03:10:11.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205502025-12-16T03:10:11.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205512025-12-16T03:10:11.241ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
205522025-12-16T03:10:11.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
205532025-12-16T03:10:11.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
205542025-12-16T03:10:11.241ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
205552025-12-16T03:10:11.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
205562025-12-16T03:10:11.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
205572025-12-16T03:10:11.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205582025-12-16T03:10:11.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205592025-12-16T03:10:11.245ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 21.525793ms iteration = 38 status = {"physical_disks_added":0}
205602025-12-16T03:10:11.245ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
205612025-12-16T03:10:11.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
205622025-12-16T03:10:11.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
205632025-12-16T03:10:11.246ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
205642025-12-16T03:10:11.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
205652025-12-16T03:10:11.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
205662025-12-16T03:10:11.247ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
205672025-12-16T03:10:11.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
205682025-12-16T03:10:11.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
205692025-12-16T03:10:11.249ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205702025-12-16T03:10:11.251ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
205712025-12-16T03:10:11.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
205722025-12-16T03:10:11.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
205732025-12-16T03:10:11.253ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205742025-12-16T03:10:11.255ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
205752025-12-16T03:10:11.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
205762025-12-16T03:10:11.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
205772025-12-16T03:10:11.257ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
205782025-12-16T03:10:11.257ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 33.334003ms iteration = 18 status = {"blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","inventory_collection_id":"4cf7dcb3-4693-4eaf-af85-9f40dc98c412","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
205792025-12-16T03:10:11.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
205802025-12-16T03:10:11.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
205812025-12-16T03:10:11.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
205822025-12-16T03:10:11.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
205832025-12-16T03:10:11.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205842025-12-16T03:10:11.295ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
205852025-12-16T03:10:11.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
205862025-12-16T03:10:11.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
205872025-12-16T03:10:11.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
205882025-12-16T03:10:11.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
205892025-12-16T03:10:11.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205902025-12-16T03:10:11.302ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
205912025-12-16T03:10:11.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
205922025-12-16T03:10:11.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
205932025-12-16T03:10:11.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205942025-12-16T03:10:11.307ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
205952025-12-16T03:10:11.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
205962025-12-16T03:10:11.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
205972025-12-16T03:10:11.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
205982025-12-16T03:10:11.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
205992025-12-16T03:10:11.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206002025-12-16T03:10:11.313ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
206012025-12-16T03:10:11.313ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206022025-12-16T03:10:11.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206032025-12-16T03:10:11.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206042025-12-16T03:10:11.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206052025-12-16T03:10:11.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206062025-12-16T03:10:11.320ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
206072025-12-16T03:10:11.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206082025-12-16T03:10:11.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206092025-12-16T03:10:11.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206102025-12-16T03:10:11.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206112025-12-16T03:10:11.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206122025-12-16T03:10:11.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
206132025-12-16T03:10:11.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206142025-12-16T03:10:11.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206152025-12-16T03:10:11.352ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206162025-12-16T03:10:11.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206172025-12-16T03:10:11.354ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206182025-12-16T03:10:11.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206192025-12-16T03:10:11.356ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206202025-12-16T03:10:11.359ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
206212025-12-16T03:10:11.359ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206222025-12-16T03:10:11.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206232025-12-16T03:10:11.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206242025-12-16T03:10:11.364ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
206252025-12-16T03:10:11.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206262025-12-16T03:10:11.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206272025-12-16T03:10:11.366ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206282025-12-16T03:10:11.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206292025-12-16T03:10:11.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206302025-12-16T03:10:11.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206312025-12-16T03:10:11.369ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206322025-12-16T03:10:11.372ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
206332025-12-16T03:10:11.372ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206342025-12-16T03:10:11.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206352025-12-16T03:10:11.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206362025-12-16T03:10:11.377ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
206372025-12-16T03:10:11.377ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206382025-12-16T03:10:11.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206392025-12-16T03:10:11.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206402025-12-16T03:10:11.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206412025-12-16T03:10:11.380ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206422025-12-16T03:10:11.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206432025-12-16T03:10:11.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206442025-12-16T03:10:11.385ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
206452025-12-16T03:10:11.386ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206462025-12-16T03:10:11.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206472025-12-16T03:10:11.387ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206482025-12-16T03:10:11.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206492025-12-16T03:10:11.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206502025-12-16T03:10:11.392ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
206512025-12-16T03:10:11.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206522025-12-16T03:10:11.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206532025-12-16T03:10:11.394ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206542025-12-16T03:10:11.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206552025-12-16T03:10:11.396ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206562025-12-16T03:10:11.399ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
206572025-12-16T03:10:11.399ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206582025-12-16T03:10:11.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206592025-12-16T03:10:11.401ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206602025-12-16T03:10:11.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206612025-12-16T03:10:11.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206622025-12-16T03:10:11.405ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
206632025-12-16T03:10:11.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206642025-12-16T03:10:11.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206652025-12-16T03:10:11.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206662025-12-16T03:10:11.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206672025-12-16T03:10:11.408ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206682025-12-16T03:10:11.411ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
206692025-12-16T03:10:11.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206702025-12-16T03:10:11.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206712025-12-16T03:10:11.413ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206722025-12-16T03:10:11.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206732025-12-16T03:10:11.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206742025-12-16T03:10:11.417ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
206752025-12-16T03:10:11.417ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206762025-12-16T03:10:11.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206772025-12-16T03:10:11.419ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
206782025-12-16T03:10:11.419ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
206792025-12-16T03:10:11.419ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = f55805aa-1430-4526-8694-c756c00f41f6 parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
206802025-12-16T03:10:11.419ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206812025-12-16T03:10:11.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206822025-12-16T03:10:11.420ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206832025-12-16T03:10:11.423ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
206842025-12-16T03:10:11.423ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
206852025-12-16T03:10:11.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
206862025-12-16T03:10:11.425ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
206872025-12-16T03:10:11.425ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
206882025-12-16T03:10:11.425ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 201.878188ms iteration = 29 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
206892025-12-16T03:10:14.319ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 134 reason = Timeout
206902025-12-16T03:10:14.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
206912025-12-16T03:10:14.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
206922025-12-16T03:10:14.321ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206932025-12-16T03:10:14.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
206942025-12-16T03:10:14.324ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
206952025-12-16T03:10:14.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
206962025-12-16T03:10:14.326ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
206972025-12-16T03:10:14.326ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.271452ms iteration = 134 status = {"config_updated":false}
206982025-12-16T03:10:19.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
206992025-12-16T03:10:19.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
207002025-12-16T03:10:19.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
207012025-12-16T03:10:19.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
207022025-12-16T03:10:19.319ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 135 reason = Timeout
207032025-12-16T03:10:19.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
207042025-12-16T03:10:19.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
207052025-12-16T03:10:19.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
207062025-12-16T03:10:19.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
207072025-12-16T03:10:19.321ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207082025-12-16T03:10:19.321ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
207092025-12-16T03:10:19.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207102025-12-16T03:10:19.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
207112025-12-16T03:10:19.323ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.866909ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
207122025-12-16T03:10:19.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
207132025-12-16T03:10:19.325ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
207142025-12-16T03:10:19.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
207152025-12-16T03:10:19.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
207162025-12-16T03:10:19.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
207172025-12-16T03:10:19.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
207182025-12-16T03:10:19.327ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
207192025-12-16T03:10:19.327ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.770679ms iteration = 135 status = {"config_updated":false}
207202025-12-16T03:10:19.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
207212025-12-16T03:10:19.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
207222025-12-16T03:10:19.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207232025-12-16T03:10:19.331ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
207242025-12-16T03:10:19.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
207252025-12-16T03:10:19.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
207262025-12-16T03:10:19.348ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
207272025-12-16T03:10:19.348ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.402123ms iteration = 68 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
207282025-12-16T03:10:21.062ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59463 req_id = 41b1121a-ccec-4809-be9a-ff4d1edd5087 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending version = 1.0.0
207292025-12-16T03:10:21.062ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59463 req_id = 41b1121a-ccec-4809-be9a-ff4d1edd5087 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
207302025-12-16T03:10:21.063ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59463 req_id = 41b1121a-ccec-4809-be9a-ff4d1edd5087 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
207312025-12-16T03:10:21.065ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 3043 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59463 req_id = 41b1121a-ccec-4809-be9a-ff4d1edd5087 response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?sort_by=id_ascending
207322025-12-16T03:10:21.066ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59463 req_id = a63f7c0d-25d3-48b3-9d0d-eb7ad7724752 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D version = 1.0.0
207332025-12-16T03:10:21.066ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59463 req_id = a63f7c0d-25d3-48b3-9d0d-eb7ad7724752 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
207342025-12-16T03:10:21.066ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59463 req_id = a63f7c0d-25d3-48b3-9d0d-eb7ad7724752 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
207352025-12-16T03:10:21.068ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 2209 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:59463 req_id = a63f7c0d-25d3-48b3-9d0d-eb7ad7724752 response_code = 200 uri = /metrics/collectors/e5b81e4f-90b9-43ba-ba56-584fd121e601/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTc0MjQ5NS0zNmRjLTQzMmEtOWQ3MS0wMjU2YWU5MmEyNTYifX0%3D
207362025-12-16T03:10:21.070ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 262 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 96827e82-0278-4cde-a7d4-0c30a61ae77c response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
207372025-12-16T03:10:24.318ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 46 reason = Timeout
207382025-12-16T03:10:24.318ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
207392025-12-16T03:10:24.318ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 60 reason = Timeout
207402025-12-16T03:10:24.318ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
207412025-12-16T03:10:24.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
207422025-12-16T03:10:24.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
207432025-12-16T03:10:24.318ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 46 reason = Timeout
207442025-12-16T03:10:24.318ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
207452025-12-16T03:10:24.318ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 136 reason = Timeout
207462025-12-16T03:10:24.318ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
207472025-12-16T03:10:24.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
207482025-12-16T03:10:24.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
207492025-12-16T03:10:24.319ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
207502025-12-16T03:10:24.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
207512025-12-16T03:10:24.320ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207522025-12-16T03:10:24.320ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207532025-12-16T03:10:24.320ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207542025-12-16T03:10:24.320ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207552025-12-16T03:10:24.323ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
207562025-12-16T03:10:24.323ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
207572025-12-16T03:10:24.323ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
207582025-12-16T03:10:24.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
207592025-12-16T03:10:24.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
207602025-12-16T03:10:24.324ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
207612025-12-16T03:10:24.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
207622025-12-16T03:10:24.325ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.812516ms iteration = 46 status = "NoSitrep"
207632025-12-16T03:10:24.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
207642025-12-16T03:10:24.325ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
207652025-12-16T03:10:24.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
207662025-12-16T03:10:24.325ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
207672025-12-16T03:10:24.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
207682025-12-16T03:10:24.326ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 4cf7dcb3-4693-4eaf-af85-9f40dc98c412 original_time_started = 2025-12-16 03:10:10.645891 UTC
207692025-12-16T03:10:24.326ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.677597ms iteration = 60 status = {"Loaded":{"collection_id":"4cf7dcb3-4693-4eaf-af85-9f40dc98c412","time_loaded":"2025-12-16T03:10:24.318290711Z","time_started":"2025-12-16T03:10:10.645891Z"}}
207702025-12-16T03:10:24.326ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
207712025-12-16T03:10:24.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
207722025-12-16T03:10:24.327ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
207732025-12-16T03:10:24.327ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.382836ms iteration = 136 status = {"config_updated":false}
207742025-12-16T03:10:24.327ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207752025-12-16T03:10:24.329ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
207762025-12-16T03:10:24.329ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
207772025-12-16T03:10:24.329ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.99918ms iteration = 46 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
207782025-12-16T03:10:29.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
207792025-12-16T03:10:29.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
207802025-12-16T03:10:29.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
207812025-12-16T03:10:29.319ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 137 reason = Timeout
207822025-12-16T03:10:29.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
207832025-12-16T03:10:29.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
207842025-12-16T03:10:29.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
207852025-12-16T03:10:29.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
207862025-12-16T03:10:29.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
207872025-12-16T03:10:29.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207882025-12-16T03:10:29.321ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
207892025-12-16T03:10:29.322ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207902025-12-16T03:10:29.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
207912025-12-16T03:10:29.323ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.152502ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
207922025-12-16T03:10:29.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
207932025-12-16T03:10:29.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
207942025-12-16T03:10:29.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
207952025-12-16T03:10:29.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
207962025-12-16T03:10:29.326ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
207972025-12-16T03:10:29.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
207982025-12-16T03:10:29.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
207992025-12-16T03:10:29.327ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
208002025-12-16T03:10:29.327ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.434007ms iteration = 137 status = {"config_updated":false}
208012025-12-16T03:10:29.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
208022025-12-16T03:10:29.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208032025-12-16T03:10:29.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
208042025-12-16T03:10:29.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
208052025-12-16T03:10:29.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
208062025-12-16T03:10:29.344ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
208072025-12-16T03:10:29.344ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.887615ms iteration = 69 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
208082025-12-16T03:10:31.070ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 413 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 58cd804c-4cf5-4ae5-8757-861c30c8d65b response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
208092025-12-16T03:10:34.319ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 138 reason = Timeout
208102025-12-16T03:10:34.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
208112025-12-16T03:10:34.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
208122025-12-16T03:10:34.322ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208132025-12-16T03:10:34.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
208142025-12-16T03:10:34.325ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
208152025-12-16T03:10:34.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
208162025-12-16T03:10:34.327ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
208172025-12-16T03:10:34.327ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.767528ms iteration = 138 status = {"config_updated":false}
208182025-12-16T03:10:39.177ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
208192025-12-16T03:10:39.177ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 24 reason = Timeout
208202025-12-16T03:10:39.177ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
208212025-12-16T03:10:39.177ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
208222025-12-16T03:10:39.177ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
208232025-12-16T03:10:39.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
208242025-12-16T03:10:39.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
208252025-12-16T03:10:39.180ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1646, tv_nsec: 788046344 } })
208262025-12-16T03:10:39.180ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1646, tv_nsec: 788045694 } })
208272025-12-16T03:10:39.181ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.042511ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
208282025-12-16T03:10:39.181ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
208292025-12-16T03:10:39.182ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1646, tv_nsec: 789654485 } })
208302025-12-16T03:10:39.259ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
208312025-12-16T03:10:39.260ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 82.719208ms iteration = 24 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
208322025-12-16T03:10:39.262ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
208332025-12-16T03:10:39.262ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
208342025-12-16T03:10:39.262ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "56fea7b3-b7db-44ef-b1bb-a8e2810b9262", "content-length": "26", "date": "Tue, 16 Dec 2025 03:10:39 GMT"} })
208352025-12-16T03:10:39.262ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
208362025-12-16T03:10:39.264ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
208372025-12-16T03:10:39.265ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1646, tv_nsec: 872857659 } })
208382025-12-16T03:10:39.303ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 125.230996ms iteration = 24 status = {}
208392025-12-16T03:10:39.318ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
208402025-12-16T03:10:39.318ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 47 reason = Timeout
208412025-12-16T03:10:39.318ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
208422025-12-16T03:10:39.318ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 61 reason = Timeout
208432025-12-16T03:10:39.318ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
208442025-12-16T03:10:39.318ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
208452025-12-16T03:10:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
208462025-12-16T03:10:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
208472025-12-16T03:10:39.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
208482025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 24 reason = Timeout
208492025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 36 reason = Timeout
208502025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 15.04\u{b5}s iteration = 24 status = {"disabled":true,"errors":[],"sps":[]}
208512025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 47 reason = Timeout
208522025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
208532025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
208542025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 36 reason = Timeout
208552025-12-16T03:10:39.319ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
208562025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 39 reason = Timeout
208572025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
208582025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
208592025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
208602025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
208612025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
208622025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
208632025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
208642025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
208652025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 24 reason = Timeout
208662025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
208672025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
208682025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
208692025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
208702025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
208712025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
208722025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
208732025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 139 reason = Timeout
208742025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
208752025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
208762025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
208772025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
208782025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
208792025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
208802025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
208812025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
208822025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
208832025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
208842025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Timeout
208852025-12-16T03:10:39.319ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
208862025-12-16T03:10:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
208872025-12-16T03:10:39.320ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208882025-12-16T03:10:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
208892025-12-16T03:10:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
208902025-12-16T03:10:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
208912025-12-16T03:10:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
208922025-12-16T03:10:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
208932025-12-16T03:10:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
208942025-12-16T03:10:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
208952025-12-16T03:10:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
208962025-12-16T03:10:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
208972025-12-16T03:10:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
208982025-12-16T03:10:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
208992025-12-16T03:10:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
209002025-12-16T03:10:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
209012025-12-16T03:10:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
209022025-12-16T03:10:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
209032025-12-16T03:10:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
209042025-12-16T03:10:39.320ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209052025-12-16T03:10:39.321ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209062025-12-16T03:10:39.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
209072025-12-16T03:10:39.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
209082025-12-16T03:10:39.322ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209092025-12-16T03:10:39.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209102025-12-16T03:10:39.322ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209112025-12-16T03:10:39.324ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 4.290584ms iteration = 24 status = {"errors":[],"finish_invoked_ok":[]}
209122025-12-16T03:10:39.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
209132025-12-16T03:10:39.324ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
209142025-12-16T03:10:39.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
209152025-12-16T03:10:39.325ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
209162025-12-16T03:10:39.325ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
209172025-12-16T03:10:39.325ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209182025-12-16T03:10:39.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
209192025-12-16T03:10:39.325ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209202025-12-16T03:10:39.326ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
209212025-12-16T03:10:39.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
209222025-12-16T03:10:39.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
209232025-12-16T03:10:39.327ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
209242025-12-16T03:10:39.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
209252025-12-16T03:10:39.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
209262025-12-16T03:10:39.328ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
209272025-12-16T03:10:39.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
209282025-12-16T03:10:39.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
209292025-12-16T03:10:39.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
209302025-12-16T03:10:39.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
209312025-12-16T03:10:39.330ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
209322025-12-16T03:10:39.330ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
209332025-12-16T03:10:39.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
209342025-12-16T03:10:39.331ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 13.210658ms iteration = 47 status = "NoSitrep"
209352025-12-16T03:10:39.331ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209362025-12-16T03:10:39.331ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1646, tv_nsec: 938882496 } })
209372025-12-16T03:10:39.331ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
209382025-12-16T03:10:39.331ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
209392025-12-16T03:10:39.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
209402025-12-16T03:10:39.332ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1646, tv_nsec: 939797677 } })
209412025-12-16T03:10:39.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
209422025-12-16T03:10:39.332ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
209432025-12-16T03:10:39.332ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
209442025-12-16T03:10:39.332ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209452025-12-16T03:10:39.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
209462025-12-16T03:10:39.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
209472025-12-16T03:10:39.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
209482025-12-16T03:10:39.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
209492025-12-16T03:10:39.334ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209502025-12-16T03:10:39.334ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209512025-12-16T03:10:39.335ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 4cf7dcb3-4693-4eaf-af85-9f40dc98c412 original_time_started = 2025-12-16 03:10:10.645891 UTC
209522025-12-16T03:10:39.335ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 17.231158ms iteration = 61 status = {"Loaded":{"collection_id":"4cf7dcb3-4693-4eaf-af85-9f40dc98c412","time_loaded":"2025-12-16T03:10:39.318099646Z","time_started":"2025-12-16T03:10:10.645891Z"}}
209532025-12-16T03:10:39.335ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209542025-12-16T03:10:39.336ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
209552025-12-16T03:10:39.336ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
209562025-12-16T03:10:39.336ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
209572025-12-16T03:10:39.336ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
209582025-12-16T03:10:39.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
209592025-12-16T03:10:39.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
209602025-12-16T03:10:39.338ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
209612025-12-16T03:10:39.338ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
209622025-12-16T03:10:39.338ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
209632025-12-16T03:10:39.338ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
209642025-12-16T03:10:39.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
209652025-12-16T03:10:39.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
209662025-12-16T03:10:39.338ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 19.206233ms iteration = 24 status = 0
209672025-12-16T03:10:39.338ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
209682025-12-16T03:10:39.338ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 19.122593ms iteration = 139 status = {"config_updated":false}
209692025-12-16T03:10:39.339ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
209702025-12-16T03:10:39.339ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
209712025-12-16T03:10:39.339ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
209722025-12-16T03:10:39.339ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
209732025-12-16T03:10:39.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
209742025-12-16T03:10:39.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
209752025-12-16T03:10:39.341ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
209762025-12-16T03:10:39.341ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 21.693705ms iteration = 36 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
209772025-12-16T03:10:39.364ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 44.865639ms iteration = 39 status = {"physical_disks_added":0}
209782025-12-16T03:10:39.367ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
209792025-12-16T03:10:39.367ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 47.852946ms iteration = 70 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
209802025-12-16T03:10:39.369ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209812025-12-16T03:10:39.371ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
209822025-12-16T03:10:39.371ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
209832025-12-16T03:10:39.371ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
209842025-12-16T03:10:39.371ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c8c44c2-c073-4dc4-a053-f6b75048a3df", "content-length": "26", "date": "Tue, 16 Dec 2025 03:10:39 GMT"} })
209852025-12-16T03:10:39.371ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
209862025-12-16T03:10:39.374ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
209872025-12-16T03:10:39.374ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
209882025-12-16T03:10:39.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
209892025-12-16T03:10:39.375ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
209902025-12-16T03:10:39.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
209912025-12-16T03:10:39.376ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
209922025-12-16T03:10:39.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
209932025-12-16T03:10:39.377ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209942025-12-16T03:10:39.379ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
209952025-12-16T03:10:39.379ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
209962025-12-16T03:10:39.379ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 60.351065ms iteration = 47 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
209972025-12-16T03:10:39.381ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 61.56697ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
209982025-12-16T03:10:39.381ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
209992025-12-16T03:10:39.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
210002025-12-16T03:10:39.383ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 63.516385ms iteration = 24 status = {"errors":[],"requests_created_ok":[]}
210012025-12-16T03:10:39.384ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
210022025-12-16T03:10:39.384ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
210032025-12-16T03:10:39.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
210042025-12-16T03:10:39.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
210052025-12-16T03:10:39.386ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 66.538603ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
210062025-12-16T03:10:39.386ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
210072025-12-16T03:10:39.386ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 66.877307ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
210082025-12-16T03:10:39.389ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
210092025-12-16T03:10:39.389ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
210102025-12-16T03:10:39.389ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
210112025-12-16T03:10:39.390ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
210122025-12-16T03:10:39.390ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
210132025-12-16T03:10:39.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
210142025-12-16T03:10:39.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
210152025-12-16T03:10:39.395ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
210162025-12-16T03:10:39.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
210172025-12-16T03:10:39.398ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
210182025-12-16T03:10:39.398ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 78.579375ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
210192025-12-16T03:10:39.406ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
210202025-12-16T03:10:39.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
210212025-12-16T03:10:39.408ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
210222025-12-16T03:10:39.412ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
210232025-12-16T03:10:39.412ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
210242025-12-16T03:10:39.412ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1647, tv_nsec: 19868512 } })
210252025-12-16T03:10:39.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
210262025-12-16T03:10:39.414ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
210272025-12-16T03:10:39.417ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
210282025-12-16T03:10:39.417ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
210292025-12-16T03:10:39.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
210302025-12-16T03:10:39.426ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 106.734392ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
210312025-12-16T03:10:39.432ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
210322025-12-16T03:10:39.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
210332025-12-16T03:10:39.434ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
210342025-12-16T03:10:39.434ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
210352025-12-16T03:10:39.434ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
210362025-12-16T03:10:39.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
210372025-12-16T03:10:39.437ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
210382025-12-16T03:10:39.437ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
210392025-12-16T03:10:39.437ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 117.449918ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
210402025-12-16T03:10:39.444ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1647, tv_nsec: 52181191 } })
210412025-12-16T03:10:39.486ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
210422025-12-16T03:10:39.486ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 166.332737ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
210432025-12-16T03:10:39.486ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
210442025-12-16T03:10:39.488ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
210452025-12-16T03:10:39.488ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
210462025-12-16T03:10:39.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
210472025-12-16T03:10:39.491ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
210482025-12-16T03:10:39.491ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
210492025-12-16T03:10:39.494ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "bcd12881-588f-4de0-8f88-08d7151797e2", "content-length": "372", "date": "Tue, 16 Dec 2025 03:10:39 GMT"} })
210502025-12-16T03:10:39.494ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
210512025-12-16T03:10:39.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
210522025-12-16T03:10:39.501ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
210532025-12-16T03:10:39.501ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
210542025-12-16T03:10:39.502ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
210552025-12-16T03:10:39.502ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
210562025-12-16T03:10:39.502ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
210572025-12-16T03:10:39.502ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
210582025-12-16T03:10:39.502ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
210592025-12-16T03:10:39.502ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
210602025-12-16T03:10:39.502ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
210612025-12-16T03:10:39.502ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
210622025-12-16T03:10:39.503ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
210632025-12-16T03:10:39.503ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
210642025-12-16T03:10:39.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
210652025-12-16T03:10:39.506ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
210662025-12-16T03:10:39.506ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
210672025-12-16T03:10:39.509ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
210682025-12-16T03:10:39.509ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
210692025-12-16T03:10:39.510ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "4cf11d35-b44c-4fc5-ab5e-b286e329664b", "date": "Tue, 16 Dec 2025 03:10:39 GMT"} })
210702025-12-16T03:10:39.510ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
210712025-12-16T03:10:39.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
210722025-12-16T03:10:39.513ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
210732025-12-16T03:10:39.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
210742025-12-16T03:10:39.516ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
210752025-12-16T03:10:39.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
210762025-12-16T03:10:39.519ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
210772025-12-16T03:10:39.519ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
210782025-12-16T03:10:39.525ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "391c083c-aab1-42a7-af50-560570504330", "date": "Tue, 16 Dec 2025 03:10:39 GMT"} })
210792025-12-16T03:10:39.527ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 207.684731ms iteration = 36 status = {}
210802025-12-16T03:10:39.545ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
210812025-12-16T03:10:39.549ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "f32064a1-bcf5-4297-8b89-abde99676cbb", "content-length": "691", "date": "Tue, 16 Dec 2025 03:10:39 GMT"} })
210822025-12-16T03:10:39.549ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
210832025-12-16T03:10:39.549ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
210842025-12-16T03:10:39.549ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
210852025-12-16T03:10:39.551ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 231.565694ms iteration = 24 status = {}
210862025-12-16T03:10:39.689ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "49b7aa18-ee5b-480a-9083-23564fa1ecab", "date": "Tue, 16 Dec 2025 03:10:39 GMT"} })
210872025-12-16T03:10:39.689ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
210882025-12-16T03:10:39.689ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
210892025-12-16T03:10:39.689ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "20c616c8-a067-458e-ae15-aecf2115033d", "content-length": "682", "date": "Tue, 16 Dec 2025 03:10:39 GMT"} })
210902025-12-16T03:10:39.689ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
210912025-12-16T03:10:39.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
210922025-12-16T03:10:39.693ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
210932025-12-16T03:10:39.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
210942025-12-16T03:10:39.697ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
210952025-12-16T03:10:39.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
210962025-12-16T03:10:39.699ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
210972025-12-16T03:10:39.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
210982025-12-16T03:10:39.706ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 388.515943ms iteration = 24 status = {}
210992025-12-16T03:10:41.070ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 277 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = a3513e76-4ebf-4ac7-8572-0ad65a4a7e1f response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
211002025-12-16T03:10:44.319ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 140 reason = Timeout
211012025-12-16T03:10:44.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
211022025-12-16T03:10:44.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
211032025-12-16T03:10:44.321ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211042025-12-16T03:10:44.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
211052025-12-16T03:10:44.324ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
211062025-12-16T03:10:44.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
211072025-12-16T03:10:44.326ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
211082025-12-16T03:10:44.326ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.916838ms iteration = 140 status = {"config_updated":false}
211092025-12-16T03:10:49.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
211102025-12-16T03:10:49.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
211112025-12-16T03:10:49.319ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 141 reason = Timeout
211122025-12-16T03:10:49.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
211132025-12-16T03:10:49.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
211142025-12-16T03:10:49.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
211152025-12-16T03:10:49.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
211162025-12-16T03:10:49.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
211172025-12-16T03:10:49.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
211182025-12-16T03:10:49.321ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211192025-12-16T03:10:49.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211202025-12-16T03:10:49.322ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
211212025-12-16T03:10:49.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
211222025-12-16T03:10:49.324ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.815991ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
211232025-12-16T03:10:49.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
211242025-12-16T03:10:49.325ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
211252025-12-16T03:10:49.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
211262025-12-16T03:10:49.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
211272025-12-16T03:10:49.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
211282025-12-16T03:10:49.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
211292025-12-16T03:10:49.327ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
211302025-12-16T03:10:49.327ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.674528ms iteration = 141 status = {"config_updated":false}
211312025-12-16T03:10:49.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
211322025-12-16T03:10:49.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
211332025-12-16T03:10:49.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211342025-12-16T03:10:49.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
211352025-12-16T03:10:49.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
211362025-12-16T03:10:49.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
211372025-12-16T03:10:49.346ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
211382025-12-16T03:10:49.346ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.279916ms iteration = 71 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
211392025-12-16T03:10:51.069ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 305 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 80425581-9ceb-4bce-85fb-a4ae02c600d8 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
211402025-12-16T03:10:54.318ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 48 reason = Timeout
211412025-12-16T03:10:54.318ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 62 reason = Timeout
211422025-12-16T03:10:54.318ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
211432025-12-16T03:10:54.318ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
211442025-12-16T03:10:54.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
211452025-12-16T03:10:54.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
211462025-12-16T03:10:54.319ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 48 reason = Timeout
211472025-12-16T03:10:54.319ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 142 reason = Timeout
211482025-12-16T03:10:54.319ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
211492025-12-16T03:10:54.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
211502025-12-16T03:10:54.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
211512025-12-16T03:10:54.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
211522025-12-16T03:10:54.320ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
211532025-12-16T03:10:54.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
211542025-12-16T03:10:54.321ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211552025-12-16T03:10:54.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211562025-12-16T03:10:54.322ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211572025-12-16T03:10:54.323ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211582025-12-16T03:10:54.325ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
211592025-12-16T03:10:54.325ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
211602025-12-16T03:10:54.325ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
211612025-12-16T03:10:54.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
211622025-12-16T03:10:54.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
211632025-12-16T03:10:54.326ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
211642025-12-16T03:10:54.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
211652025-12-16T03:10:54.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
211662025-12-16T03:10:54.326ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
211672025-12-16T03:10:54.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
211682025-12-16T03:10:54.327ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 8.472337ms iteration = 48 status = "NoSitrep"
211692025-12-16T03:10:54.328ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 4cf7dcb3-4693-4eaf-af85-9f40dc98c412 original_time_started = 2025-12-16 03:10:10.645891 UTC
211702025-12-16T03:10:54.328ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 9.51592ms iteration = 62 status = {"Loaded":{"collection_id":"4cf7dcb3-4693-4eaf-af85-9f40dc98c412","time_loaded":"2025-12-16T03:10:54.318869973Z","time_started":"2025-12-16T03:10:10.645891Z"}}
211712025-12-16T03:10:54.328ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
211722025-12-16T03:10:54.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
211732025-12-16T03:10:54.328ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
211742025-12-16T03:10:54.328ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 9.578852ms iteration = 142 status = {"config_updated":false}
211752025-12-16T03:10:54.329ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
211762025-12-16T03:10:54.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
211772025-12-16T03:10:54.331ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211782025-12-16T03:10:54.332ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
211792025-12-16T03:10:54.333ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
211802025-12-16T03:10:54.333ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 13.844756ms iteration = 48 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
211812025-12-16T03:10:59.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
211822025-12-16T03:10:59.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
211832025-12-16T03:10:59.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
211842025-12-16T03:10:59.319ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 143 reason = Timeout
211852025-12-16T03:10:59.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
211862025-12-16T03:10:59.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
211872025-12-16T03:10:59.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
211882025-12-16T03:10:59.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
211892025-12-16T03:10:59.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
211902025-12-16T03:10:59.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211912025-12-16T03:10:59.321ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211922025-12-16T03:10:59.321ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
211932025-12-16T03:10:59.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
211942025-12-16T03:10:59.324ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.555768ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
211952025-12-16T03:10:59.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
211962025-12-16T03:10:59.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
211972025-12-16T03:10:59.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
211982025-12-16T03:10:59.326ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
211992025-12-16T03:10:59.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
212002025-12-16T03:10:59.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
212012025-12-16T03:10:59.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
212022025-12-16T03:10:59.328ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
212032025-12-16T03:10:59.328ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.562979ms iteration = 143 status = {"config_updated":false}
212042025-12-16T03:10:59.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
212052025-12-16T03:10:59.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212062025-12-16T03:10:59.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
212072025-12-16T03:10:59.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
212082025-12-16T03:10:59.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
212092025-12-16T03:10:59.347ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
212102025-12-16T03:10:59.347ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.208967ms iteration = 72 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
212112025-12-16T03:11:01.071ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 455 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 97218e9c-efa1-4db1-8603-21fc8d36b218 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
212122025-12-16T03:11:04.319ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 144 reason = Timeout
212132025-12-16T03:11:04.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
212142025-12-16T03:11:04.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
212152025-12-16T03:11:04.322ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212162025-12-16T03:11:04.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
212172025-12-16T03:11:04.325ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
212182025-12-16T03:11:04.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
212192025-12-16T03:11:04.327ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
212202025-12-16T03:11:04.327ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.802619ms iteration = 144 status = {"config_updated":false}
212212025-12-16T03:11:09.177ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
212222025-12-16T03:11:09.177ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
212232025-12-16T03:11:09.177ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
212242025-12-16T03:11:09.177ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
212252025-12-16T03:11:09.177ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
212262025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
212272025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
212282025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
212292025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
212302025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
212312025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
212322025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 25 reason = Timeout
212332025-12-16T03:11:09.178ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-12-16 03:01:09.178024475 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
212342025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
212352025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
212362025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
212372025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
212382025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
212392025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
212402025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
212412025-12-16T03:11:09.178ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
212422025-12-16T03:11:09.178ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 86c7f3bb-27c2-416b-af6f-cfd364cc926e step component = ExternalNetworking step id = Ensure
212432025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
212442025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
212452025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
212462025-12-16T03:11:09.178ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1a45cd23-3d0a-43eb-adce-62c7b712baf1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1676, tv_nsec: 785757975 } })
212472025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
212482025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
212492025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
212502025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
212512025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
212522025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
212532025-12-16T03:11:09.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
212542025-12-16T03:11:09.178ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("49a51a59-9919-48ae-8182-4070a8430160.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6af60386-8771-4401-9d4d-e93774e519f0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 1676, tv_nsec: 785987048 } })
212552025-12-16T03:11:09.179ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
212562025-12-16T03:11:09.179ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.430468ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
212572025-12-16T03:11:09.179ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
212582025-12-16T03:11:09.179ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.63445ms iteration = 13 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
212592025-12-16T03:11:09.179ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
212602025-12-16T03:11:09.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
212612025-12-16T03:11:09.182ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212622025-12-16T03:11:09.182ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
212632025-12-16T03:11:09.182ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1676, tv_nsec: 789671665 } })
212642025-12-16T03:11:09.182ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1676, tv_nsec: 789829857 } })
212652025-12-16T03:11:09.182ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
212662025-12-16T03:11:09.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
212672025-12-16T03:11:09.183ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212682025-12-16T03:11:09.183ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1676, tv_nsec: 790368624 } })
212692025-12-16T03:11:09.183ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-12-16 03:01:09.178024475 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
212702025-12-16T03:11:09.183ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.580931ms iteration = 13 status = {"expiration":"2025-12-16T03:01:09.178024475Z","pruned":[]}
212712025-12-16T03:11:09.184ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
212722025-12-16T03:11:09.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
212732025-12-16T03:11:09.185ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212742025-12-16T03:11:09.185ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
212752025-12-16T03:11:09.185ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
212762025-12-16T03:11:09.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
212772025-12-16T03:11:09.186ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 8.467457ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
212782025-12-16T03:11:09.186ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
212792025-12-16T03:11:09.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
212802025-12-16T03:11:09.188ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212812025-12-16T03:11:09.189ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
212822025-12-16T03:11:09.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
212832025-12-16T03:11:09.189ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
212842025-12-16T03:11:09.191ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
212852025-12-16T03:11:09.191ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
212862025-12-16T03:11:09.191ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
212872025-12-16T03:11:09.191ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
212882025-12-16T03:11:09.191ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212892025-12-16T03:11:09.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
212902025-12-16T03:11:09.192ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
212912025-12-16T03:11:09.193ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
212922025-12-16T03:11:09.194ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
212932025-12-16T03:11:09.194ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
212942025-12-16T03:11:09.194ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
212952025-12-16T03:11:09.194ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
212962025-12-16T03:11:09.194ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212972025-12-16T03:11:09.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
212982025-12-16T03:11:09.195ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
212992025-12-16T03:11:09.196ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213002025-12-16T03:11:09.197ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
213012025-12-16T03:11:09.197ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
213022025-12-16T03:11:09.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
213032025-12-16T03:11:09.198ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
213042025-12-16T03:11:09.200ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
213052025-12-16T03:11:09.201ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
213062025-12-16T03:11:09.202ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
213072025-12-16T03:11:09.202ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
213082025-12-16T03:11:09.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
213092025-12-16T03:11:09.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
213102025-12-16T03:11:09.203ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
213112025-12-16T03:11:09.203ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
213122025-12-16T03:11:09.203ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
213132025-12-16T03:11:09.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
213142025-12-16T03:11:09.204ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213152025-12-16T03:11:09.204ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
213162025-12-16T03:11:09.205ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
213172025-12-16T03:11:09.205ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
213182025-12-16T03:11:09.205ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
213192025-12-16T03:11:09.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
213202025-12-16T03:11:09.207ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
213212025-12-16T03:11:09.207ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
213222025-12-16T03:11:09.207ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
213232025-12-16T03:11:09.207ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
213242025-12-16T03:11:09.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
213252025-12-16T03:11:09.208ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
213262025-12-16T03:11:09.208ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
213272025-12-16T03:11:09.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
213282025-12-16T03:11:09.208ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
213292025-12-16T03:11:09.208ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213302025-12-16T03:11:09.210ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
213312025-12-16T03:11:09.210ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213322025-12-16T03:11:09.211ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
213332025-12-16T03:11:09.214ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
213342025-12-16T03:11:09.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
213352025-12-16T03:11:09.214ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
213362025-12-16T03:11:09.214ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
213372025-12-16T03:11:09.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
213382025-12-16T03:11:09.216ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
213392025-12-16T03:11:09.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
213402025-12-16T03:11:09.217ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
213412025-12-16T03:11:09.217ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 86c7f3bb-27c2-416b-af6f-cfd364cc926e step component = SledList step id = Fetch
213422025-12-16T03:11:09.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
213432025-12-16T03:11:09.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
213442025-12-16T03:11:09.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
213452025-12-16T03:11:09.218ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
213462025-12-16T03:11:09.218ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
213472025-12-16T03:11:09.218ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-12-16 02:59:18.154440 UTC dns_group = external generation = 2
213482025-12-16T03:11:09.218ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 40.090978ms iteration = 13 status = {"generation":2}
213492025-12-16T03:11:09.219ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 30 reason = Timeout
213502025-12-16T03:11:09.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213512025-12-16T03:11:09.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213522025-12-16T03:11:09.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213532025-12-16T03:11:09.221ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
213542025-12-16T03:11:09.221ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
213552025-12-16T03:11:09.221ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
213562025-12-16T03:11:09.221ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
213572025-12-16T03:11:09.221ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
213582025-12-16T03:11:09.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
213592025-12-16T03:11:09.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
213602025-12-16T03:11:09.221ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
213612025-12-16T03:11:09.221ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
213622025-12-16T03:11:09.221ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
213632025-12-16T03:11:09.221ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
213642025-12-16T03:11:09.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
213652025-12-16T03:11:09.221ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
213662025-12-16T03:11:09.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
213672025-12-16T03:11:09.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
213682025-12-16T03:11:09.221ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
213692025-12-16T03:11:09.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
213702025-12-16T03:11:09.221ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
213712025-12-16T03:11:09.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
213722025-12-16T03:11:09.222ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
213732025-12-16T03:11:09.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
213742025-12-16T03:11:09.223ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
213752025-12-16T03:11:09.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
213762025-12-16T03:11:09.224ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
213772025-12-16T03:11:09.224ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213782025-12-16T03:11:09.224ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213792025-12-16T03:11:09.227ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213802025-12-16T03:11:09.227ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213812025-12-16T03:11:09.228ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
213822025-12-16T03:11:09.228ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
213832025-12-16T03:11:09.228ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-12-16 02:59:16.791641 UTC dns_group = internal generation = 1
213842025-12-16T03:11:09.228ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 51.048147ms iteration = 13 status = {"generation":1}
213852025-12-16T03:11:09.229ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
213862025-12-16T03:11:09.229ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
213872025-12-16T03:11:09.229ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
213882025-12-16T03:11:09.229ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
213892025-12-16T03:11:09.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
213902025-12-16T03:11:09.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
213912025-12-16T03:11:09.229ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213922025-12-16T03:11:09.232ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
213932025-12-16T03:11:09.232ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
213942025-12-16T03:11:09.232ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
213952025-12-16T03:11:09.232ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
213962025-12-16T03:11:09.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
213972025-12-16T03:11:09.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
213982025-12-16T03:11:09.234ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
213992025-12-16T03:11:09.234ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
214002025-12-16T03:11:09.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
214012025-12-16T03:11:09.234ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214022025-12-16T03:11:09.236ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 42 reason = Timeout
214032025-12-16T03:11:09.236ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
214042025-12-16T03:11:09.236ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 13 reason = Timeout
214052025-12-16T03:11:09.236ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
214062025-12-16T03:11:09.236ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 13 reason = Timeout
214072025-12-16T03:11:09.236ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 25.07\u{b5}s iteration = 42 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
214082025-12-16T03:11:09.236ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
214092025-12-16T03:11:09.236ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 13 reason = Timeout
214102025-12-16T03:11:09.236ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
214112025-12-16T03:11:09.237ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
214122025-12-16T03:11:09.237ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
214132025-12-16T03:11:09.237ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
214142025-12-16T03:11:09.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
214152025-12-16T03:11:09.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
214162025-12-16T03:11:09.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
214172025-12-16T03:11:09.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
214182025-12-16T03:11:09.238ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
214192025-12-16T03:11:09.238ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
214202025-12-16T03:11:09.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
214212025-12-16T03:11:09.238ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214222025-12-16T03:11:09.241ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
214232025-12-16T03:11:09.241ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 4.140013ms iteration = 13 status = {"by_rx":{},"error":null}
214242025-12-16T03:11:09.241ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
214252025-12-16T03:11:09.241ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
214262025-12-16T03:11:09.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
214272025-12-16T03:11:09.242ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214282025-12-16T03:11:09.243ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 6.29672ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
214292025-12-16T03:11:09.245ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
214302025-12-16T03:11:09.246ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
214312025-12-16T03:11:09.246ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214322025-12-16T03:11:09.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214332025-12-16T03:11:09.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
214342025-12-16T03:11:09.246ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
214352025-12-16T03:11:09.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
214362025-12-16T03:11:09.247ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 86c7f3bb-27c2-416b-af6f-cfd364cc926e step component = DeployNexusRecords step id = Ensure
214372025-12-16T03:11:09.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
214382025-12-16T03:11:09.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
214392025-12-16T03:11:09.249ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214402025-12-16T03:11:09.250ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
214412025-12-16T03:11:09.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
214422025-12-16T03:11:09.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
214432025-12-16T03:11:09.250ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 14.115259ms iteration = 13 status = {"dispatched":[],"errors":[],"glob_version":"212.0.0","globs_reprocessed":{},"no_receivers":[]}
214442025-12-16T03:11:09.251ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
214452025-12-16T03:11:09.251ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
214462025-12-16T03:11:09.251ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 30.890962ms iteration = 6 status = {"nsuccess":5}
214472025-12-16T03:11:09.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
214482025-12-16T03:11:09.252ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
214492025-12-16T03:11:09.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
214502025-12-16T03:11:09.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
214512025-12-16T03:11:09.253ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214522025-12-16T03:11:09.253ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
214532025-12-16T03:11:09.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
214542025-12-16T03:11:09.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
214552025-12-16T03:11:09.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
214562025-12-16T03:11:09.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214572025-12-16T03:11:09.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214582025-12-16T03:11:09.257ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
214592025-12-16T03:11:09.257ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
214602025-12-16T03:11:09.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
214612025-12-16T03:11:09.259ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
214622025-12-16T03:11:09.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
214632025-12-16T03:11:09.261ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
214642025-12-16T03:11:09.261ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
214652025-12-16T03:11:09.261ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
214662025-12-16T03:11:09.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
214672025-12-16T03:11:09.262ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214682025-12-16T03:11:09.265ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
214692025-12-16T03:11:09.266ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
214702025-12-16T03:11:09.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
214712025-12-16T03:11:09.266ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214722025-12-16T03:11:09.270ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
214732025-12-16T03:11:09.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
214742025-12-16T03:11:09.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
214752025-12-16T03:11:09.292ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-12-16T02:59:05.020237Z, time_modified: 2025-12-16T02:59:05.020237Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
214762025-12-16T03:11:09.292ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 114.374389ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]}},"default_endpoint":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
214772025-12-16T03:11:09.295ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
214782025-12-16T03:11:09.295ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
214792025-12-16T03:11:09.295ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 75.359094ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
214802025-12-16T03:11:09.312ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 86c7f3bb-27c2-416b-af6f-cfd364cc926e step component = SledAgent step id = Ensure
214812025-12-16T03:11:09.313ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
214822025-12-16T03:11:09.313ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
214832025-12-16T03:11:09.313ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 93.323942ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
214842025-12-16T03:11:09.318ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
214852025-12-16T03:11:09.318ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 49 reason = Timeout
214862025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 63 reason = Timeout
214872025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
214882025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
214892025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
214902025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
214912025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
214922025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
214932025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 25 reason = Timeout
214942025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
214952025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
214962025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 12.38\u{b5}s iteration = 25 status = {"disabled":true,"errors":[],"sps":[]}
214972025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
214982025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 25 reason = Timeout
214992025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 40 reason = Timeout
215002025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
215012025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
215022025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
215032025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
215042025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
215052025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
215062025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
215072025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
215082025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
215092025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 37 reason = Timeout
215102025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
215112025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
215122025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
215132025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
215142025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
215152025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
215162025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 49 reason = Timeout
215172025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 37 reason = Timeout
215182025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
215192025-12-16T03:11:09.319ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
215202025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
215212025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
215222025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
215232025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
215242025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
215252025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 145 reason = Timeout
215262025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
215272025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
215282025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
215292025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
215302025-12-16T03:11:09.319ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
215312025-12-16T03:11:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
215322025-12-16T03:11:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
215332025-12-16T03:11:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
215342025-12-16T03:11:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
215352025-12-16T03:11:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
215362025-12-16T03:11:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
215372025-12-16T03:11:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
215382025-12-16T03:11:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
215392025-12-16T03:11:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
215402025-12-16T03:11:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
215412025-12-16T03:11:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
215422025-12-16T03:11:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
215432025-12-16T03:11:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
215442025-12-16T03:11:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
215452025-12-16T03:11:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
215462025-12-16T03:11:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
215472025-12-16T03:11:09.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
215482025-12-16T03:11:09.321ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
215492025-12-16T03:11:09.321ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215502025-12-16T03:11:09.322ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
215512025-12-16T03:11:09.322ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215522025-12-16T03:11:09.322ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215532025-12-16T03:11:09.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
215542025-12-16T03:11:09.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215552025-12-16T03:11:09.322ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215562025-12-16T03:11:09.322ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215572025-12-16T03:11:09.323ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
215582025-12-16T03:11:09.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
215592025-12-16T03:11:09.324ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215602025-12-16T03:11:09.324ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215612025-12-16T03:11:09.324ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215622025-12-16T03:11:09.325ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
215632025-12-16T03:11:09.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
215642025-12-16T03:11:09.325ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215652025-12-16T03:11:09.326ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215662025-12-16T03:11:09.329ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 8.553618ms iteration = 25 status = {"errors":[],"finish_invoked_ok":[]}
215672025-12-16T03:11:09.329ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
215682025-12-16T03:11:09.329ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 9.903545ms iteration = 25 status = {"errors":[],"requests_created_ok":[]}
215692025-12-16T03:11:09.329ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
215702025-12-16T03:11:09.329ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
215712025-12-16T03:11:09.329ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
215722025-12-16T03:11:09.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
215732025-12-16T03:11:09.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
215742025-12-16T03:11:09.330ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
215752025-12-16T03:11:09.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
215762025-12-16T03:11:09.330ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
215772025-12-16T03:11:09.330ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
215782025-12-16T03:11:09.330ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
215792025-12-16T03:11:09.330ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
215802025-12-16T03:11:09.330ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
215812025-12-16T03:11:09.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
215822025-12-16T03:11:09.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
215832025-12-16T03:11:09.330ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
215842025-12-16T03:11:09.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
215852025-12-16T03:11:09.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
215862025-12-16T03:11:09.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
215872025-12-16T03:11:09.331ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
215882025-12-16T03:11:09.331ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
215892025-12-16T03:11:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
215902025-12-16T03:11:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
215912025-12-16T03:11:09.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
215922025-12-16T03:11:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
215932025-12-16T03:11:09.332ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
215942025-12-16T03:11:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
215952025-12-16T03:11:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
215962025-12-16T03:11:09.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
215972025-12-16T03:11:09.333ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
215982025-12-16T03:11:09.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
215992025-12-16T03:11:09.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
216002025-12-16T03:11:09.333ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
216012025-12-16T03:11:09.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
216022025-12-16T03:11:09.334ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 14.680606ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
216032025-12-16T03:11:09.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
216042025-12-16T03:11:09.334ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
216052025-12-16T03:11:09.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
216062025-12-16T03:11:09.334ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216072025-12-16T03:11:09.335ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 16.157145ms iteration = 49 status = "NoSitrep"
216082025-12-16T03:11:09.335ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
216092025-12-16T03:11:09.335ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
216102025-12-16T03:11:09.335ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
216112025-12-16T03:11:09.335ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
216122025-12-16T03:11:09.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
216132025-12-16T03:11:09.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
216142025-12-16T03:11:09.336ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 4cf7dcb3-4693-4eaf-af85-9f40dc98c412 original_time_started = 2025-12-16 03:10:10.645891 UTC
216152025-12-16T03:11:09.336ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216162025-12-16T03:11:09.336ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 17.115197ms iteration = 63 status = {"Loaded":{"collection_id":"4cf7dcb3-4693-4eaf-af85-9f40dc98c412","time_loaded":"2025-12-16T03:11:09.318998652Z","time_started":"2025-12-16T03:10:10.645891Z"}}
216172025-12-16T03:11:09.336ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
216182025-12-16T03:11:09.336ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
216192025-12-16T03:11:09.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216202025-12-16T03:11:09.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
216212025-12-16T03:11:09.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
216222025-12-16T03:11:09.336ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
216232025-12-16T03:11:09.336ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 17.035006ms iteration = 145 status = {"config_updated":false}
216242025-12-16T03:11:09.337ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
216252025-12-16T03:11:09.337ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 17.880986ms iteration = 25 status = 0
216262025-12-16T03:11:09.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
216272025-12-16T03:11:09.337ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1676, tv_nsec: 944963053 } })
216282025-12-16T03:11:09.338ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
216292025-12-16T03:11:09.338ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
216302025-12-16T03:11:09.338ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
216312025-12-16T03:11:09.338ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1676, tv_nsec: 945983066 } })
216322025-12-16T03:11:09.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
216332025-12-16T03:11:09.339ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216342025-12-16T03:11:09.340ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
216352025-12-16T03:11:09.340ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216362025-12-16T03:11:09.340ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 21.162698ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
216372025-12-16T03:11:09.344ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
216382025-12-16T03:11:09.344ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
216392025-12-16T03:11:09.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
216402025-12-16T03:11:09.344ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
216412025-12-16T03:11:09.344ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216422025-12-16T03:11:09.345ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
216432025-12-16T03:11:09.345ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 25.573074ms iteration = 49 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
216442025-12-16T03:11:09.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216452025-12-16T03:11:09.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216462025-12-16T03:11:09.350ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
216472025-12-16T03:11:09.350ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
216482025-12-16T03:11:09.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
216492025-12-16T03:11:09.352ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216502025-12-16T03:11:09.356ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
216512025-12-16T03:11:09.356ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216522025-12-16T03:11:09.357ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
216532025-12-16T03:11:09.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216542025-12-16T03:11:09.360ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216552025-12-16T03:11:09.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216562025-12-16T03:11:09.363ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216572025-12-16T03:11:09.367ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
216582025-12-16T03:11:09.367ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216592025-12-16T03:11:09.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216602025-12-16T03:11:09.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216612025-12-16T03:11:09.375ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
216622025-12-16T03:11:09.375ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216632025-12-16T03:11:09.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216642025-12-16T03:11:09.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216652025-12-16T03:11:09.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216662025-12-16T03:11:09.383ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
216672025-12-16T03:11:09.383ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 63.288052ms iteration = 73 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
216682025-12-16T03:11:09.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216692025-12-16T03:11:09.385ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
216702025-12-16T03:11:09.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
216712025-12-16T03:11:09.388ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 68.161764ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
216722025-12-16T03:11:09.389ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 69.578892ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
216732025-12-16T03:11:09.390ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
216742025-12-16T03:11:09.390ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216752025-12-16T03:11:09.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216762025-12-16T03:11:09.393ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216772025-12-16T03:11:09.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216782025-12-16T03:11:09.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216792025-12-16T03:11:09.400ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
216802025-12-16T03:11:09.400ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216812025-12-16T03:11:09.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216822025-12-16T03:11:09.400ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
216832025-12-16T03:11:09.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
216842025-12-16T03:11:09.403ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
216852025-12-16T03:11:09.403ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216862025-12-16T03:11:09.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
216872025-12-16T03:11:09.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216882025-12-16T03:11:09.404ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216892025-12-16T03:11:09.408ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216902025-12-16T03:11:09.408ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
216912025-12-16T03:11:09.408ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
216922025-12-16T03:11:09.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
216932025-12-16T03:11:09.410ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
216942025-12-16T03:11:09.410ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
216952025-12-16T03:11:09.411ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8eff4505-8d42-47a8-a618-034edc768cd9", "content-length": "26", "date": "Tue, 16 Dec 2025 03:11:09 GMT"} })
216962025-12-16T03:11:09.411ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
216972025-12-16T03:11:09.412ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
216982025-12-16T03:11:09.412ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216992025-12-16T03:11:09.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217002025-12-16T03:11:09.413ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 94.071023ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
217012025-12-16T03:11:09.416ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
217022025-12-16T03:11:09.416ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
217032025-12-16T03:11:09.416ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 97.046059ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
217042025-12-16T03:11:09.418ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
217052025-12-16T03:11:09.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
217062025-12-16T03:11:09.425ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
217072025-12-16T03:11:09.426ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1677, tv_nsec: 33544955 } })
217082025-12-16T03:11:09.439ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217092025-12-16T03:11:09.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217102025-12-16T03:11:09.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217112025-12-16T03:11:09.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217122025-12-16T03:11:09.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217132025-12-16T03:11:09.449ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
217142025-12-16T03:11:09.450ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217152025-12-16T03:11:09.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217162025-12-16T03:11:09.469ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
217172025-12-16T03:11:09.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
217182025-12-16T03:11:09.472ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 152.508362ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
217192025-12-16T03:11:09.477ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
217202025-12-16T03:11:09.477ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 158.037011ms iteration = 37 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
217212025-12-16T03:11:09.519ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
217222025-12-16T03:11:09.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
217232025-12-16T03:11:09.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217242025-12-16T03:11:09.523ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
217252025-12-16T03:11:09.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217262025-12-16T03:11:09.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217272025-12-16T03:11:09.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217282025-12-16T03:11:09.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217292025-12-16T03:11:09.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217302025-12-16T03:11:09.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217312025-12-16T03:11:09.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217322025-12-16T03:11:09.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
217332025-12-16T03:11:09.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217342025-12-16T03:11:09.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217352025-12-16T03:11:09.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217362025-12-16T03:11:09.543ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
217372025-12-16T03:11:09.543ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217382025-12-16T03:11:09.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217392025-12-16T03:11:09.546ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217402025-12-16T03:11:09.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217412025-12-16T03:11:09.546ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217422025-12-16T03:11:09.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217432025-12-16T03:11:09.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217442025-12-16T03:11:09.553ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 233.806902ms iteration = 40 status = {"physical_disks_added":0}
217452025-12-16T03:11:09.553ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
217462025-12-16T03:11:09.554ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
217472025-12-16T03:11:09.554ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "4a600c7e-68c0-4552-b110-e04482dd4845", "date": "Tue, 16 Dec 2025 03:11:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
217482025-12-16T03:11:09.554ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
217492025-12-16T03:11:09.556ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
217502025-12-16T03:11:09.557ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "21f35361-45e2-46f5-8a75-c92661632c17", "date": "Tue, 16 Dec 2025 03:11:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
217512025-12-16T03:11:09.557ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
217522025-12-16T03:11:09.560ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 382.254233ms iteration = 25 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
217532025-12-16T03:11:09.561ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
217542025-12-16T03:11:09.561ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 383.168305ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
217552025-12-16T03:11:09.598ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217562025-12-16T03:11:09.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217572025-12-16T03:11:09.599ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217582025-12-16T03:11:09.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217592025-12-16T03:11:09.601ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217602025-12-16T03:11:09.606ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
217612025-12-16T03:11:09.611ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217622025-12-16T03:11:09.612ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217632025-12-16T03:11:09.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217642025-12-16T03:11:09.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217652025-12-16T03:11:09.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217662025-12-16T03:11:09.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
217672025-12-16T03:11:09.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217682025-12-16T03:11:09.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217692025-12-16T03:11:09.625ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217702025-12-16T03:11:09.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217712025-12-16T03:11:09.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217722025-12-16T03:11:09.629ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
217732025-12-16T03:11:09.631ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "04a9ef12-cf9f-41a1-ba6a-ef765a1f27f2", "date": "Tue, 16 Dec 2025 03:11:09 GMT"} }) sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
217742025-12-16T03:11:09.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
217752025-12-16T03:11:09.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217762025-12-16T03:11:09.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217772025-12-16T03:11:09.633ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 86c7f3bb-27c2-416b-af6f-cfd364cc926e step component = FirewallRules step id = Ensure
217782025-12-16T03:11:09.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
217792025-12-16T03:11:09.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
217802025-12-16T03:11:09.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217812025-12-16T03:11:09.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217822025-12-16T03:11:09.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
217832025-12-16T03:11:09.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217842025-12-16T03:11:09.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
217852025-12-16T03:11:09.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
217862025-12-16T03:11:09.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
217872025-12-16T03:11:09.639ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
217882025-12-16T03:11:09.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217892025-12-16T03:11:09.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217902025-12-16T03:11:09.639ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
217912025-12-16T03:11:09.639ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "f1b70d76-6090-4fc9-a43e-899d84037ded", "date": "Tue, 16 Dec 2025 03:11:09 GMT"} })
217922025-12-16T03:11:09.639ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
217932025-12-16T03:11:09.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
217942025-12-16T03:11:09.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
217952025-12-16T03:11:09.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
217962025-12-16T03:11:09.642ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
217972025-12-16T03:11:09.642ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 405.340505ms iteration = 13 status = {"errors":[],"probes_by_sled":{"b1c76b61-acee-4be6-b582-973c938d6b6e":0}}
217982025-12-16T03:11:09.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
217992025-12-16T03:11:09.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218002025-12-16T03:11:09.642ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
218012025-12-16T03:11:09.642ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 422.062727ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
218022025-12-16T03:11:09.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218032025-12-16T03:11:09.644ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218042025-12-16T03:11:09.644ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218052025-12-16T03:11:09.644ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
218062025-12-16T03:11:09.644ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
218072025-12-16T03:11:09.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218082025-12-16T03:11:09.644ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a557db42-34de-4ff9-a583-41d9b50cd7e1", "content-length": "26", "date": "Tue, 16 Dec 2025 03:11:09 GMT"} })
218092025-12-16T03:11:09.644ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
218102025-12-16T03:11:09.646ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218112025-12-16T03:11:09.646ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
218122025-12-16T03:11:09.646ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
218132025-12-16T03:11:09.646ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
218142025-12-16T03:11:09.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
218152025-12-16T03:11:09.648ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
218162025-12-16T03:11:09.648ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
218172025-12-16T03:11:09.648ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 554d7667-f55d-4cb1-a356-eac7bff20462 parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
218182025-12-16T03:11:09.648ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
218192025-12-16T03:11:09.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
218202025-12-16T03:11:09.650ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
218212025-12-16T03:11:09.650ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218222025-12-16T03:11:09.650ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218232025-12-16T03:11:09.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218242025-12-16T03:11:09.651ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218252025-12-16T03:11:09.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218262025-12-16T03:11:09.653ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218272025-12-16T03:11:09.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218282025-12-16T03:11:09.654ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
218292025-12-16T03:11:09.654ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
218302025-12-16T03:11:09.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
218312025-12-16T03:11:09.655ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218322025-12-16T03:11:09.659ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
218332025-12-16T03:11:09.659ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
218342025-12-16T03:11:09.660ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 2 limit = 5000 usage_percent = 0
218352025-12-16T03:11:09.660ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
218362025-12-16T03:11:09.660ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e90af10-b216-4217-b85a-81449df2c779", "content-length": "372", "date": "Tue, 16 Dec 2025 03:11:09 GMT"} })
218372025-12-16T03:11:09.660ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
218382025-12-16T03:11:09.660ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 441.160869ms iteration = 30 status = {"Unchanged":{"blueprint_count":2,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
218392025-12-16T03:11:09.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
218402025-12-16T03:11:09.664ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
218412025-12-16T03:11:09.664ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218422025-12-16T03:11:09.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218432025-12-16T03:11:09.667ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218442025-12-16T03:11:09.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218452025-12-16T03:11:09.669ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218462025-12-16T03:11:09.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218472025-12-16T03:11:09.671ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218482025-12-16T03:11:09.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218492025-12-16T03:11:09.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218502025-12-16T03:11:09.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218512025-12-16T03:11:09.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218522025-12-16T03:11:09.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218532025-12-16T03:11:09.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218542025-12-16T03:11:09.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218552025-12-16T03:11:09.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218562025-12-16T03:11:09.680ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 502.431696ms iteration = 25 status = {}
218572025-12-16T03:11:09.681ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
218582025-12-16T03:11:09.685ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
218592025-12-16T03:11:09.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218602025-12-16T03:11:09.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218612025-12-16T03:11:09.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218622025-12-16T03:11:09.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218632025-12-16T03:11:09.689ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218642025-12-16T03:11:09.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218652025-12-16T03:11:09.692ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218662025-12-16T03:11:09.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218672025-12-16T03:11:09.694ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
218682025-12-16T03:11:09.695ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218692025-12-16T03:11:09.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
218702025-12-16T03:11:09.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218712025-12-16T03:11:09.696ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218722025-12-16T03:11:09.697ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218732025-12-16T03:11:09.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218742025-12-16T03:11:09.699ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
218752025-12-16T03:11:09.699ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
218762025-12-16T03:11:09.699ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218772025-12-16T03:11:09.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
218782025-12-16T03:11:09.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218792025-12-16T03:11:09.700ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218802025-12-16T03:11:09.701ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218812025-12-16T03:11:09.703ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
218822025-12-16T03:11:09.703ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
218832025-12-16T03:11:09.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
218842025-12-16T03:11:09.706ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
218852025-12-16T03:11:09.706ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218862025-12-16T03:11:09.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218872025-12-16T03:11:09.708ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218882025-12-16T03:11:09.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218892025-12-16T03:11:09.709ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218902025-12-16T03:11:09.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218912025-12-16T03:11:09.711ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218922025-12-16T03:11:09.712ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
218932025-12-16T03:11:09.712ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
218942025-12-16T03:11:09.713ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "1be5eea2-4f91-4f80-aa84-8e2b5d095acf", "date": "Tue, 16 Dec 2025 03:11:09 GMT"} })
218952025-12-16T03:11:09.713ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
218962025-12-16T03:11:09.713ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
218972025-12-16T03:11:09.715ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
218982025-12-16T03:11:09.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
218992025-12-16T03:11:09.716ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
219002025-12-16T03:11:09.716ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
219012025-12-16T03:11:09.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
219022025-12-16T03:11:09.717ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1677, tv_nsec: 325022388 } })
219032025-12-16T03:11:09.719ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
219042025-12-16T03:11:09.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
219052025-12-16T03:11:09.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
219062025-12-16T03:11:09.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
219072025-12-16T03:11:09.722ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
219082025-12-16T03:11:09.722ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
219092025-12-16T03:11:09.723ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "eef5ae48-c041-4f73-8721-bf3bac43ebc8", "date": "Tue, 16 Dec 2025 03:11:09 GMT"} })
219102025-12-16T03:11:09.723ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
219112025-12-16T03:11:09.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
219122025-12-16T03:11:09.725ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 405.500198ms iteration = 37 status = {}
219132025-12-16T03:11:09.725ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 38 reason = Dependency
219142025-12-16T03:11:09.725ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
219152025-12-16T03:11:09.725ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
219162025-12-16T03:11:09.725ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
219172025-12-16T03:11:09.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
219182025-12-16T03:11:09.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
219192025-12-16T03:11:09.727ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219202025-12-16T03:11:09.727ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
219212025-12-16T03:11:09.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
219222025-12-16T03:11:09.729ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
219232025-12-16T03:11:09.729ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
219242025-12-16T03:11:09.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
219252025-12-16T03:11:09.730ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
219262025-12-16T03:11:09.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
219272025-12-16T03:11:09.731ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219282025-12-16T03:11:09.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
219292025-12-16T03:11:09.731ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
219302025-12-16T03:11:09.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
219312025-12-16T03:11:09.732ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "c4c65847-09fc-4290-a1b7-ba29b6c5bb7c", "date": "Tue, 16 Dec 2025 03:11:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
219322025-12-16T03:11:09.732ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
219332025-12-16T03:11:09.733ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
219342025-12-16T03:11:09.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
219352025-12-16T03:11:09.733ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
219362025-12-16T03:11:09.733ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
219372025-12-16T03:11:09.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
219382025-12-16T03:11:09.733ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "fb816492-e51d-4b44-a0c2-d76bfc4237f4", "date": "Tue, 16 Dec 2025 03:11:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
219392025-12-16T03:11:09.733ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
219402025-12-16T03:11:09.734ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219412025-12-16T03:11:09.735ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "5cd5adb1-0519-439f-8040-7c27db59af69", "date": "Tue, 16 Dec 2025 03:11:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
219422025-12-16T03:11:09.735ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
219432025-12-16T03:11:09.738ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
219442025-12-16T03:11:09.738ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 560.278938ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
219452025-12-16T03:11:09.740ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
219462025-12-16T03:11:09.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
219472025-12-16T03:11:09.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
219482025-12-16T03:11:09.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
219492025-12-16T03:11:09.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
219502025-12-16T03:11:09.743ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
219512025-12-16T03:11:09.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
219522025-12-16T03:11:09.744ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219532025-12-16T03:11:09.750ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
219542025-12-16T03:11:09.750ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
219552025-12-16T03:11:09.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
219562025-12-16T03:11:09.757ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], Name("nexus"): [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
219572025-12-16T03:11:09.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
219582025-12-16T03:11:09.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
219592025-12-16T03:11:09.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219602025-12-16T03:11:09.761ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
219612025-12-16T03:11:09.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
219622025-12-16T03:11:09.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
219632025-12-16T03:11:09.763ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
219642025-12-16T03:11:09.763ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
219652025-12-16T03:11:09.763ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
219662025-12-16T03:11:09.776ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
219672025-12-16T03:11:09.779ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "2059757e-7bc4-44f9-a544-0f4fcd0509c0", "content-length": "691", "date": "Tue, 16 Dec 2025 03:11:09 GMT"} })
219682025-12-16T03:11:09.779ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
219692025-12-16T03:11:09.779ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
219702025-12-16T03:11:09.779ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
219712025-12-16T03:11:09.781ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 461.472857ms iteration = 25 status = {}
219722025-12-16T03:11:09.781ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1677, tv_nsec: 388877557 } })
219732025-12-16T03:11:09.813ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
219742025-12-16T03:11:09.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
219752025-12-16T03:11:09.816ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
219762025-12-16T03:11:09.816ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
219772025-12-16T03:11:09.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
219782025-12-16T03:11:09.827ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), time_created: 2025-12-16T02:59:15.195809Z, time_modified: 2025-12-16T02:59:15.195809Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::1:0, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
219792025-12-16T03:11:09.827ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
219802025-12-16T03:11:09.831ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
219812025-12-16T03:11:09.832ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
219822025-12-16T03:11:09.832ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
219832025-12-16T03:11:09.832ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
219842025-12-16T03:11:09.832ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
219852025-12-16T03:11:09.832ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
219862025-12-16T03:11:09.832ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
219872025-12-16T03:11:09.832ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
219882025-12-16T03:11:09.833ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
219892025-12-16T03:11:09.833ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
219902025-12-16T03:11:09.833ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
219912025-12-16T03:11:09.833ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
219922025-12-16T03:11:09.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
219932025-12-16T03:11:09.836ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
219942025-12-16T03:11:09.836ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
219952025-12-16T03:11:09.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
219962025-12-16T03:11:09.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
219972025-12-16T03:11:09.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
219982025-12-16T03:11:09.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
219992025-12-16T03:11:09.873ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
220002025-12-16T03:11:09.873ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
220012025-12-16T03:11:09.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220022025-12-16T03:11:09.874ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "d46f0d32-fa3b-4f0d-8205-2329560f3424", "content-length": "372", "date": "Tue, 16 Dec 2025 03:11:09 GMT"} })
220032025-12-16T03:11:09.874ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
220042025-12-16T03:11:09.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
220052025-12-16T03:11:09.876ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
220062025-12-16T03:11:09.876ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
220072025-12-16T03:11:09.877ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "7b569f09-0412-45bc-9742-2ec01836ebdd", "date": "Tue, 16 Dec 2025 03:11:09 GMT"} })
220082025-12-16T03:11:09.878ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
220092025-12-16T03:11:09.878ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 86c7f3bb-27c2-416b-af6f-cfd364cc926e step component = Dns step id = Ensure
220102025-12-16T03:11:09.878ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
220112025-12-16T03:11:09.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220122025-12-16T03:11:09.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220132025-12-16T03:11:09.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220142025-12-16T03:11:09.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220152025-12-16T03:11:09.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220162025-12-16T03:11:09.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
220172025-12-16T03:11:09.929ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
220182025-12-16T03:11:09.929ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
220192025-12-16T03:11:09.929ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
220202025-12-16T03:11:09.929ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220212025-12-16T03:11:09.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220222025-12-16T03:11:09.931ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220232025-12-16T03:11:09.935ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
220242025-12-16T03:11:09.935ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
220252025-12-16T03:11:09.936ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
220262025-12-16T03:11:09.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220272025-12-16T03:11:09.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220282025-12-16T03:11:09.937ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "0a75c77c-48bb-41f7-91a6-5fdc488caa8a", "date": "Tue, 16 Dec 2025 03:11:09 GMT"} })
220292025-12-16T03:11:09.937ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
220302025-12-16T03:11:09.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
220312025-12-16T03:11:09.940ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
220322025-12-16T03:11:09.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
220332025-12-16T03:11:09.942ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
220342025-12-16T03:11:09.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
220352025-12-16T03:11:09.946ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
220362025-12-16T03:11:09.946ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
220372025-12-16T03:11:09.952ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "d6f10d2b-4f13-4c3a-a9c6-b8420753b435", "date": "Tue, 16 Dec 2025 03:11:09 GMT"} })
220382025-12-16T03:11:09.954ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 229.375396ms iteration = 38 status = {}
220392025-12-16T03:11:09.961ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "82e20fa5-0639-4246-b0ec-c86a9ee83e80", "date": "Tue, 16 Dec 2025 03:11:09 GMT"} })
220402025-12-16T03:11:09.961ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
220412025-12-16T03:11:09.961ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
220422025-12-16T03:11:09.962ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca2f75e2-89d3-4a11-b47b-e835c2e15ea6", "content-length": "682", "date": "Tue, 16 Dec 2025 03:11:09 GMT"} })
220432025-12-16T03:11:09.962ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
220442025-12-16T03:11:09.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
220452025-12-16T03:11:09.965ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
220462025-12-16T03:11:09.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
220472025-12-16T03:11:09.969ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
220482025-12-16T03:11:09.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
220492025-12-16T03:11:09.971ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
220502025-12-16T03:11:09.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
220512025-12-16T03:11:09.975ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
220522025-12-16T03:11:09.975ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
220532025-12-16T03:11:09.975ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220542025-12-16T03:11:09.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220552025-12-16T03:11:09.977ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220562025-12-16T03:11:09.979ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 660.732481ms iteration = 25 status = {}
220572025-12-16T03:11:09.980ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
220582025-12-16T03:11:09.980ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220592025-12-16T03:11:09.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220602025-12-16T03:11:09.983ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
220612025-12-16T03:11:09.983ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
220622025-12-16T03:11:09.983ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
220632025-12-16T03:11:09.983ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220642025-12-16T03:11:09.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220652025-12-16T03:11:09.984ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220662025-12-16T03:11:09.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220672025-12-16T03:11:09.986ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220682025-12-16T03:11:09.989ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
220692025-12-16T03:11:09.990ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
220702025-12-16T03:11:09.990ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
220712025-12-16T03:11:09.990ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
220722025-12-16T03:11:09.990ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220732025-12-16T03:11:09.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220742025-12-16T03:11:09.992ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220752025-12-16T03:11:09.995ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
220762025-12-16T03:11:09.995ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220772025-12-16T03:11:09.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220782025-12-16T03:11:10.041ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
220792025-12-16T03:11:10.041ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
220802025-12-16T03:11:10.041ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220812025-12-16T03:11:10.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220822025-12-16T03:11:10.043ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220832025-12-16T03:11:10.047ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
220842025-12-16T03:11:10.047ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220852025-12-16T03:11:10.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220862025-12-16T03:11:10.049ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
220872025-12-16T03:11:10.049ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
220882025-12-16T03:11:10.049ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220892025-12-16T03:11:10.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220902025-12-16T03:11:10.051ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220912025-12-16T03:11:10.053ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
220922025-12-16T03:11:10.053ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220932025-12-16T03:11:10.053ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220942025-12-16T03:11:10.056ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220952025-12-16T03:11:10.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220962025-12-16T03:11:10.056ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220972025-12-16T03:11:10.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220982025-12-16T03:11:10.058ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220992025-12-16T03:11:10.062ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
221002025-12-16T03:11:10.065ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
221012025-12-16T03:11:10.065ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
221022025-12-16T03:11:10.065ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 86c7f3bb-27c2-416b-af6f-cfd364cc926e step component = OmicronZones step id = Cleanup
221032025-12-16T03:11:10.065ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 86c7f3bb-27c2-416b-af6f-cfd364cc926e step component = OmicronZones step id = Cleanup
221042025-12-16T03:11:10.065ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 86c7f3bb-27c2-416b-af6f-cfd364cc926e step component = PhysicalDisks step id = Cleanup
221052025-12-16T03:11:10.065ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 86c7f3bb-27c2-416b-af6f-cfd364cc926e step component = Clickhouse step id = Ensure
221062025-12-16T03:11:10.065ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 86c7f3bb-27c2-416b-af6f-cfd364cc926e step component = Clickhouse step id = Ensure
221072025-12-16T03:11:10.098ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
221082025-12-16T03:11:10.102ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "8ed1424b-19e1-473f-962f-08cd26e63dc4", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
221092025-12-16T03:11:10.103ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 86c7f3bb-27c2-416b-af6f-cfd364cc926e step component = SupportBundles step id = Cleanup
221102025-12-16T03:11:10.103ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221112025-12-16T03:11:10.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221122025-12-16T03:11:10.105ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
221132025-12-16T03:11:10.106ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
221142025-12-16T03:11:10.107ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221152025-12-16T03:11:10.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221162025-12-16T03:11:10.108ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221172025-12-16T03:11:10.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221182025-12-16T03:11:10.109ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
221192025-12-16T03:11:10.112ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
221202025-12-16T03:11:10.153ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 86c7f3bb-27c2-416b-af6f-cfd364cc926e step component = OmicronZones step id = Cleanup
221212025-12-16T03:11:10.153ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
221222025-12-16T03:11:10.153ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
221232025-12-16T03:11:10.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221242025-12-16T03:11:10.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221252025-12-16T03:11:10.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
221262025-12-16T03:11:10.157ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
221272025-12-16T03:11:10.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221282025-12-16T03:11:10.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221292025-12-16T03:11:10.201ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
221302025-12-16T03:11:10.201ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(a91734db-9ac7-4793-9c32-ddbb21f8d986 (blueprint), false)
221312025-12-16T03:11:10.201ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 86c7f3bb-27c2-416b-af6f-cfd364cc926e step component = Cockroach step id = Ensure
221322025-12-16T03:11:10.201ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221332025-12-16T03:11:10.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221342025-12-16T03:11:10.215ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
221352025-12-16T03:11:10.215ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 86c7f3bb-27c2-416b-af6f-cfd364cc926e step component = MgsUpdates step id = Ensure
221362025-12-16T03:11:10.215ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 38 reason = Dependency
221372025-12-16T03:11:10.215ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 15 reason = Dependency
221382025-12-16T03:11:10.215ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
221392025-12-16T03:11:10.215ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
221402025-12-16T03:11:10.215ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.037725168s iteration = 14 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"f983c0fb-8ef7-4ee9-a952-36efb8259fc8","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"f983c0fb-8ef7-4ee9-a952-36efb8259fc8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":4310,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":39095325,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":39095285,"secs":0}},"event_index":1,"execution_id":"f983c0fb-8ef7-4ee9-a952-36efb8259fc8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":39112556,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":30065771,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":30065721,"secs":0}},"event_index":2,"execution_id":"f983c0fb-8ef7-4ee9-a952-36efb8259fc8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":69190257,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":64957863,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":64957833,"secs":0}},"event_index":3,"execution_id":"f983c0fb-8ef7-4ee9-a952-36efb8259fc8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":134168280,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":321034418,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":321034387,"secs":0}},"event_index":4,"execution_id":"f983c0fb-8ef7-4ee9-a952-36efb8259fc8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":455222887,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":245570821,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":245570791,"secs":0}},"event_index":5,"execution_id":"f983c0fb-8ef7-4ee9-a952-36efb8259fc8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":700806139,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":186600865,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":186600835,"secs":0}},"event_index":6,"execution_id":"f983c0fb-8ef7-4ee9-a952-36efb8259fc8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":887414704,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4190,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4160,"secs":0}},"event_index":7,"execution_id":"f983c0fb-8ef7-4ee9-a952-36efb8259fc8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":887430274,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2190,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2150,"secs":0}},"event_index":8,"execution_id":"f983c0fb-8ef7-4ee9-a952-36efb8259fc8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":887434634,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3920,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3870,"secs":0}},"event_index":9,"execution_id":"f983c0fb-8ef7-4ee9-a952-36efb8259fc8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":887440444,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1810,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1770,"secs":0}},"event_index":10,"execution_id":"f983c0fb-8ef7-4ee9-a952-36efb8259fc8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":887444354,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":38162924,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":38162884,"secs":0}},"event_index":11,"execution_id":"f983c0fb-8ef7-4ee9-a952-36efb8259fc8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":925608698,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":50126015,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":50125985,"secs":0}},"event_index":12,"execution_id":"f983c0fb-8ef7-4ee9-a952-36efb8259fc8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":975748693,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":47846577,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":47846537,"secs":0}},"event_index":13,"execution_id":"f983c0fb-8ef7-4ee9-a952-36efb8259fc8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":23610749,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":13652422,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":13652382,"secs":0}},"event_index":14,"execution_id":"f983c0fb-8ef7-4ee9-a952-36efb8259fc8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":37274133,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5390,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5340,"secs":0}},"event_index":15,"execution_id":"f983c0fb-8ef7-4ee9-a952-36efb8259fc8","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":37288273,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986"}
221412025-12-16T03:11:10.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
221422025-12-16T03:11:10.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
221432025-12-16T03:11:10.217ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
221442025-12-16T03:11:10.218ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
221452025-12-16T03:11:10.220ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
221462025-12-16T03:11:10.220ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
221472025-12-16T03:11:10.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
221482025-12-16T03:11:10.221ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
221492025-12-16T03:11:10.221ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
221502025-12-16T03:11:10.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
221512025-12-16T03:11:10.223ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
221522025-12-16T03:11:10.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
221532025-12-16T03:11:10.224ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
221542025-12-16T03:11:10.226ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
221552025-12-16T03:11:10.226ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
221562025-12-16T03:11:10.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
221572025-12-16T03:11:10.228ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
221582025-12-16T03:11:10.228ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 13.238428ms iteration = 38 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
221592025-12-16T03:11:10.315ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(c98952ab-e784-47f5-b72e-5889b99827e8, 0), (a51e37d5-b58a-4ed9-b46c-46ad89c1f4ad, 0), (25bc4c2d-fc88-4f83-a2be-4a590e58ead0, 0), (9731e6a4-dd37-464b-8145-974fc4f4be58, 0)] collection_id = c98952ab-e784-47f5-b72e-5889b99827e8
221602025-12-16T03:11:10.315ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
221612025-12-16T03:11:10.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
221622025-12-16T03:11:10.484ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = c98952ab-e784-47f5-b72e-5889b99827e8 file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
221632025-12-16T03:11:10.484ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
221642025-12-16T03:11:10.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
221652025-12-16T03:11:10.569ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(25bc4c2d-fc88-4f83-a2be-4a590e58ead0, 0), (9731e6a4-dd37-464b-8145-974fc4f4be58, 0), (4cf7dcb3-4693-4eaf-af85-9f40dc98c412, 0)]
221662025-12-16T03:11:10.569ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1678, tv_nsec: 176556566 } })
221672025-12-16T03:11:10.601ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("20488683-5778-4090-b36e-e6f0d80a682b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("35dbaad1-8536-44a0-8a0a-35d7c39dcfbb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b57fa226-2428-4a54-b85d-60409e28b484.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("d812a3e2-48ae-4812-a1cc-7eb7623e4e00.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::5)) }], valid_until: Instant { tv_sec: 1678, tv_nsec: 208903156 } })
221682025-12-16T03:11:10.602ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
221692025-12-16T03:11:10.602ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
221702025-12-16T03:11:10.602ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
221712025-12-16T03:11:10.603ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0e38105-d1a0-49fe-bde3-b43261156a6a", "content-length": "476", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
221722025-12-16T03:11:10.603ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
221732025-12-16T03:11:10.603ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "08085ea3-e832-4634-936b-7b6b14dce310", "content-length": "734", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
221742025-12-16T03:11:10.603ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
221752025-12-16T03:11:10.603ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4ad1f14-b38e-447a-9233-42228ac137d7", "content-length": "111", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
221762025-12-16T03:11:10.603ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
221772025-12-16T03:11:10.604ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d19bca4d-71d9-45cc-b704-e18bfbbe670b", "content-length": "111", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
221782025-12-16T03:11:10.604ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
221792025-12-16T03:11:10.604ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "72c8f8c3-5f30-4460-bc5e-af4a2261d9ec", "content-length": "163", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
221802025-12-16T03:11:10.604ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
221812025-12-16T03:11:10.604ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb08bd15-97da-4d1a-87eb-5434d08d800e", "content-length": "163", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
221822025-12-16T03:11:10.605ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
221832025-12-16T03:11:10.605ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "43c6f7f4-5118-4f15-baf8-d144dc9d2e3c", "content-length": "166", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
221842025-12-16T03:11:10.605ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
221852025-12-16T03:11:10.605ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2772713d-0679-47a9-954d-26b2537bf54e", "content-length": "166", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
221862025-12-16T03:11:10.605ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
221872025-12-16T03:11:10.605ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8952911-b800-4a98-88e1-7ad5e8ab6c51", "content-length": "702", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
221882025-12-16T03:11:10.605ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
221892025-12-16T03:11:10.606ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9f14f7f-9917-4558-8ec0-cc3568334962", "content-length": "727", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
221902025-12-16T03:11:10.606ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
221912025-12-16T03:11:10.606ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "efcb929e-a5b2-49f4-9d26-d8c1672d05f5", "content-length": "729", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
221922025-12-16T03:11:10.606ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
221932025-12-16T03:11:10.606ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "36dc7c1d-8c02-403f-8456-379af03bebd6", "content-length": "728", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
221942025-12-16T03:11:10.606ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
221952025-12-16T03:11:10.606ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a59920be-d8f7-4fb5-828b-6efa8526b5da", "content-length": "722", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
221962025-12-16T03:11:10.606ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
221972025-12-16T03:11:10.607ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a2d80c8-7fd4-46f4-a85e-ed7c5f96cf5f", "content-length": "10", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
221982025-12-16T03:11:10.607ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
221992025-12-16T03:11:10.607ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e0a6dad-9c07-405a-a3be-29d8ef9416d5", "content-length": "144", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222002025-12-16T03:11:10.607ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
222012025-12-16T03:11:10.607ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "598e8cf6-9f1a-4ac5-99ac-33b248f7ba4c", "content-length": "147", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222022025-12-16T03:11:10.607ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
222032025-12-16T03:11:10.607ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "67097b45-c385-4653-b5aa-ef8b3f956291", "content-length": "109", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222042025-12-16T03:11:10.607ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
222052025-12-16T03:11:10.608ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef2a939e-da8a-43f5-bcfb-38c41d8fbec0", "content-length": "109", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222062025-12-16T03:11:10.608ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
222072025-12-16T03:11:10.608ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f3d4e76-e280-4e12-bb29-8462b578baf6", "content-length": "163", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222082025-12-16T03:11:10.608ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
222092025-12-16T03:11:10.609ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e166a418-08c3-45c2-a5b1-a165160b56ac", "content-length": "163", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222102025-12-16T03:11:10.609ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
222112025-12-16T03:11:10.609ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f1f1b9c-95ff-4a7c-bd7d-cd5af3e44efc", "content-length": "166", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222122025-12-16T03:11:10.609ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
222132025-12-16T03:11:10.610ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "269b1695-9c93-4136-b91a-beb15e906f16", "content-length": "166", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222142025-12-16T03:11:10.610ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
222152025-12-16T03:11:10.610ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d491fcc8-6e46-4682-8e00-872bb4dddbf8", "content-length": "702", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222162025-12-16T03:11:10.610ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
222172025-12-16T03:11:10.610ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3124a0dc-7637-4c4d-b074-0adcc806e39c", "content-length": "727", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222182025-12-16T03:11:10.610ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
222192025-12-16T03:11:10.610ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4060bc25-2d1e-4c20-afcc-a86d594a9b02", "content-length": "729", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222202025-12-16T03:11:10.610ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
222212025-12-16T03:11:10.610ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "56746a83-ea80-46e2-b23a-165c99c1a374", "content-length": "728", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222222025-12-16T03:11:10.610ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
222232025-12-16T03:11:10.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
222242025-12-16T03:11:10.613ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
222252025-12-16T03:11:10.615ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
222262025-12-16T03:11:10.615ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
222272025-12-16T03:11:10.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
222282025-12-16T03:11:10.617ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
222292025-12-16T03:11:10.619ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
222302025-12-16T03:11:10.619ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
222312025-12-16T03:11:10.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
222322025-12-16T03:11:10.733ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
222332025-12-16T03:11:10.733ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
222342025-12-16T03:11:10.797ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "030202a1-1b5d-4c1e-90a5-2db8831261ef", "content-length": "95845", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222352025-12-16T03:11:10.799ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
222362025-12-16T03:11:10.799ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
222372025-12-16T03:11:10.799ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
222382025-12-16T03:11:10.799ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
222392025-12-16T03:11:10.799ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
222402025-12-16T03:11:10.799ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
222412025-12-16T03:11:10.799ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
222422025-12-16T03:11:10.799ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
222432025-12-16T03:11:10.799ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "af4db1ca-d076-4f21-87ef-f0bdacc003b1", "content-length": "64", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222442025-12-16T03:11:10.799ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
222452025-12-16T03:11:10.799ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0f0c0a9-c8b5-486e-959e-d8db84413eae", "content-length": "64", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222462025-12-16T03:11:10.799ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
222472025-12-16T03:11:10.799ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "97f5ad3f-029b-4d20-b660-e30b8d3dee8f", "content-length": "64", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222482025-12-16T03:11:10.799ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7efd5ef8-78a9-4da4-99d0-db7ad911a4e7", "content-length": "64", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222492025-12-16T03:11:10.799ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
222502025-12-16T03:11:10.799ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
222512025-12-16T03:11:10.799ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "541c16df-5ba9-41d2-a1a7-557926d0247e", "content-length": "64", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222522025-12-16T03:11:10.799ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
222532025-12-16T03:11:10.838ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "396fe667-e59c-4bef-8e47-388cedd46771", "content-length": "412855", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222542025-12-16T03:11:10.839ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "260e911a-e192-411b-8876-e3240770a81c", "content-length": "410985", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222552025-12-16T03:11:10.840ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "536a705c-a0ed-4ec7-869e-9af2260f466d", "content-length": "413553", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222562025-12-16T03:11:10.841ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "7bda723f-c124-4658-8646-21b16920a515", "content-length": "439877", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222572025-12-16T03:11:10.842ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd2ab670-ea05-414b-91e2-ece12479fafd", "content-length": "411534", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222582025-12-16T03:11:10.845ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
222592025-12-16T03:11:10.845ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
222602025-12-16T03:11:10.845ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
222612025-12-16T03:11:10.845ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
222622025-12-16T03:11:10.845ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
222632025-12-16T03:11:10.875ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c (omicron_zone)
222642025-12-16T03:11:10.875ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
222652025-12-16T03:11:10.878ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "871ee86d-a029-4fdd-8892-1c4370ddc52f", "content-length": "125", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222662025-12-16T03:11:10.880ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
222672025-12-16T03:11:10.970ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
222682025-12-16T03:11:10.970ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
222692025-12-16T03:11:10.970ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7e952cc-2cb0-4c51-8784-d3c394cf3a57", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222702025-12-16T03:11:10.970ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
222712025-12-16T03:11:10.972ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
222722025-12-16T03:11:10.972ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
222732025-12-16T03:11:10.972ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d45188a-1c7d-43d0-9a19-0fecc354bbe4", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222742025-12-16T03:11:10.972ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
222752025-12-16T03:11:10.974ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
222762025-12-16T03:11:10.974ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
222772025-12-16T03:11:10.974ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "3250f8c1-c74f-4d16-9321-083997b53a34", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:11:10 GMT"} })
222782025-12-16T03:11:10.974ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
222792025-12-16T03:11:10.975ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
222802025-12-16T03:11:10.975ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
222812025-12-16T03:11:10.977ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
222822025-12-16T03:11:10.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
222832025-12-16T03:11:10.979ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
222842025-12-16T03:11:10.982ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
222852025-12-16T03:11:10.982ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
222862025-12-16T03:11:10.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
222872025-12-16T03:11:11.070ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 277 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 15f7ed7b-c669-4275-a463-ba6db72b4c40 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
222882025-12-16T03:11:11.182ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 60390ee0-9ebf-417a-b74d-f8a1c4e08914 file = nexus/db-queries/src/db/datastore/inventory.rs:1613
222892025-12-16T03:11:11.182ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 60390ee0-9ebf-417a-b74d-f8a1c4e08914 time_started = 2025-12-16 03:11:10.602552 UTC
222902025-12-16T03:11:11.182ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 43 reason = Dependency
222912025-12-16T03:11:11.182ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
222922025-12-16T03:11:11.182ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 19.181\u{b5}s iteration = 43 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
222932025-12-16T03:11:11.182ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 64 reason = Dependency
222942025-12-16T03:11:11.182ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
222952025-12-16T03:11:11.182ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 967.365297ms iteration = 15 status = {"collection_id":"60390ee0-9ebf-417a-b74d-f8a1c4e08914","time_done":"2025-12-16 03:11:10.975926 UTC","time_started":"2025-12-16 03:11:10.602552 UTC"}
222962025-12-16T03:11:11.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
222972025-12-16T03:11:11.185ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
222982025-12-16T03:11:11.188ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
222992025-12-16T03:11:11.188ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
223002025-12-16T03:11:11.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
223012025-12-16T03:11:11.191ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
223022025-12-16T03:11:11.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
223032025-12-16T03:11:11.233ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 41 reason = Dependency
223042025-12-16T03:11:11.233ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 51.045606ms iteration = 64 status = {"Loaded":{"collection_id":"60390ee0-9ebf-417a-b74d-f8a1c4e08914","time_loaded":"2025-12-16T03:11:11.182881927Z","time_started":"2025-12-16T03:11:10.602552Z"}}
223052025-12-16T03:11:11.234ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 31 reason = Dependency
223062025-12-16T03:11:11.234ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
223072025-12-16T03:11:11.234ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 44 reason = Dependency
223082025-12-16T03:11:11.234ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
223092025-12-16T03:11:11.234ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
223102025-12-16T03:11:11.234ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 21.3\u{b5}s iteration = 44 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
223112025-12-16T03:11:11.234ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 19 reason = Dependency
223122025-12-16T03:11:11.234ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
223132025-12-16T03:11:11.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
223142025-12-16T03:11:11.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
223152025-12-16T03:11:11.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
223162025-12-16T03:11:11.235ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223172025-12-16T03:11:11.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223182025-12-16T03:11:11.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223192025-12-16T03:11:11.239ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
223202025-12-16T03:11:11.239ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
223212025-12-16T03:11:11.239ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
223222025-12-16T03:11:11.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
223232025-12-16T03:11:11.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
223242025-12-16T03:11:11.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
223252025-12-16T03:11:11.239ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
223262025-12-16T03:11:11.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
223272025-12-16T03:11:11.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
223282025-12-16T03:11:11.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223292025-12-16T03:11:11.241ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
223302025-12-16T03:11:11.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223312025-12-16T03:11:11.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
223322025-12-16T03:11:11.242ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223332025-12-16T03:11:11.243ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
223342025-12-16T03:11:11.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
223352025-12-16T03:11:11.244ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
223362025-12-16T03:11:11.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
223372025-12-16T03:11:11.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
223382025-12-16T03:11:11.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
223392025-12-16T03:11:11.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
223402025-12-16T03:11:11.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
223412025-12-16T03:11:11.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
223422025-12-16T03:11:11.246ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
223432025-12-16T03:11:11.246ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
223442025-12-16T03:11:11.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
223452025-12-16T03:11:11.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
223462025-12-16T03:11:11.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223472025-12-16T03:11:11.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223482025-12-16T03:11:11.250ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
223492025-12-16T03:11:11.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
223502025-12-16T03:11:11.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
223512025-12-16T03:11:11.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223522025-12-16T03:11:11.252ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
223532025-12-16T03:11:11.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
223542025-12-16T03:11:11.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
223552025-12-16T03:11:11.254ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
223562025-12-16T03:11:11.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
223572025-12-16T03:11:11.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
223582025-12-16T03:11:11.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223592025-12-16T03:11:11.257ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
223602025-12-16T03:11:11.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
223612025-12-16T03:11:11.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
223622025-12-16T03:11:11.259ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
223632025-12-16T03:11:11.259ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
223642025-12-16T03:11:11.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
223652025-12-16T03:11:11.261ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223662025-12-16T03:11:11.263ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
223672025-12-16T03:11:11.263ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
223682025-12-16T03:11:11.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
223692025-12-16T03:11:11.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223702025-12-16T03:11:11.267ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
223712025-12-16T03:11:11.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
223722025-12-16T03:11:11.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
223732025-12-16T03:11:11.269ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
223742025-12-16T03:11:11.269ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 35.430999ms iteration = 19 status = {"blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","inventory_collection_id":"60390ee0-9ebf-417a-b74d-f8a1c4e08914","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
223752025-12-16T03:11:11.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
223762025-12-16T03:11:11.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
223772025-12-16T03:11:11.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
223782025-12-16T03:11:11.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
223792025-12-16T03:11:11.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223802025-12-16T03:11:11.309ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
223812025-12-16T03:11:11.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
223822025-12-16T03:11:11.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
223832025-12-16T03:11:11.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
223842025-12-16T03:11:11.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
223852025-12-16T03:11:11.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223862025-12-16T03:11:11.315ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
223872025-12-16T03:11:11.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
223882025-12-16T03:11:11.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
223892025-12-16T03:11:11.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223902025-12-16T03:11:11.319ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
223912025-12-16T03:11:11.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
223922025-12-16T03:11:11.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
223932025-12-16T03:11:11.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
223942025-12-16T03:11:11.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
223952025-12-16T03:11:11.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223962025-12-16T03:11:11.326ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
223972025-12-16T03:11:11.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
223982025-12-16T03:11:11.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
223992025-12-16T03:11:11.328ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 94.108853ms iteration = 41 status = {"physical_disks_added":0}
224002025-12-16T03:11:11.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224012025-12-16T03:11:11.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224022025-12-16T03:11:11.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224032025-12-16T03:11:11.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
224042025-12-16T03:11:11.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224052025-12-16T03:11:11.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224062025-12-16T03:11:11.335ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224072025-12-16T03:11:11.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224082025-12-16T03:11:11.337ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224092025-12-16T03:11:11.340ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
224102025-12-16T03:11:11.340ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224112025-12-16T03:11:11.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224122025-12-16T03:11:11.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224132025-12-16T03:11:11.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224142025-12-16T03:11:11.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224152025-12-16T03:11:11.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224162025-12-16T03:11:11.359ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224172025-12-16T03:11:11.362ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
224182025-12-16T03:11:11.362ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224192025-12-16T03:11:11.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224202025-12-16T03:11:11.363ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224212025-12-16T03:11:11.366ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
224222025-12-16T03:11:11.366ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224232025-12-16T03:11:11.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224242025-12-16T03:11:11.369ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224252025-12-16T03:11:11.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224262025-12-16T03:11:11.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224272025-12-16T03:11:11.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224282025-12-16T03:11:11.372ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224292025-12-16T03:11:11.375ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
224302025-12-16T03:11:11.375ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224312025-12-16T03:11:11.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224322025-12-16T03:11:11.377ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224332025-12-16T03:11:11.380ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
224342025-12-16T03:11:11.380ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224352025-12-16T03:11:11.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224362025-12-16T03:11:11.383ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224372025-12-16T03:11:11.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224382025-12-16T03:11:11.383ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224392025-12-16T03:11:11.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224402025-12-16T03:11:11.385ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224412025-12-16T03:11:11.388ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
224422025-12-16T03:11:11.389ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224432025-12-16T03:11:11.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224442025-12-16T03:11:11.390ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224452025-12-16T03:11:11.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224462025-12-16T03:11:11.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224472025-12-16T03:11:11.394ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
224482025-12-16T03:11:11.396ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224492025-12-16T03:11:11.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224502025-12-16T03:11:11.398ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224512025-12-16T03:11:11.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224522025-12-16T03:11:11.400ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224532025-12-16T03:11:11.402ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
224542025-12-16T03:11:11.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224552025-12-16T03:11:11.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224562025-12-16T03:11:11.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224572025-12-16T03:11:11.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224582025-12-16T03:11:11.406ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224592025-12-16T03:11:11.409ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
224602025-12-16T03:11:11.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224612025-12-16T03:11:11.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224622025-12-16T03:11:11.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224632025-12-16T03:11:11.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224642025-12-16T03:11:11.413ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224652025-12-16T03:11:11.415ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
224662025-12-16T03:11:11.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224672025-12-16T03:11:11.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224682025-12-16T03:11:11.417ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224692025-12-16T03:11:11.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224702025-12-16T03:11:11.419ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224712025-12-16T03:11:11.421ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
224722025-12-16T03:11:11.421ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224732025-12-16T03:11:11.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224742025-12-16T03:11:11.456ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
224752025-12-16T03:11:11.456ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
224762025-12-16T03:11:11.456ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 7c7057db-9c5b-45d5-bf0f-f5f67637ac24 parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
224772025-12-16T03:11:11.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224782025-12-16T03:11:11.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224792025-12-16T03:11:11.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224802025-12-16T03:11:11.461ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
224812025-12-16T03:11:11.461ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
224822025-12-16T03:11:11.461ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
224832025-12-16T03:11:11.464ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 2 limit = 5000 usage_percent = 0
224842025-12-16T03:11:11.464ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
224852025-12-16T03:11:11.464ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 230.235537ms iteration = 31 status = {"Unchanged":{"blueprint_count":2,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
224862025-12-16T03:11:14.319ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 146 reason = Timeout
224872025-12-16T03:11:14.319ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
224882025-12-16T03:11:14.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
224892025-12-16T03:11:14.322ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224902025-12-16T03:11:14.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
224912025-12-16T03:11:14.325ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
224922025-12-16T03:11:14.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
224932025-12-16T03:11:14.327ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
224942025-12-16T03:11:14.327ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.082323ms iteration = 146 status = {"config_updated":false}
224952025-12-16T03:11:19.319ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
224962025-12-16T03:11:19.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
224972025-12-16T03:11:19.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
224982025-12-16T03:11:19.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
224992025-12-16T03:11:19.320ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 147 reason = Timeout
225002025-12-16T03:11:19.320ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
225012025-12-16T03:11:19.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
225022025-12-16T03:11:19.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
225032025-12-16T03:11:19.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
225042025-12-16T03:11:19.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225052025-12-16T03:11:19.322ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225062025-12-16T03:11:19.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
225072025-12-16T03:11:19.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
225082025-12-16T03:11:19.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
225092025-12-16T03:11:19.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
225102025-12-16T03:11:19.326ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
225112025-12-16T03:11:19.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
225122025-12-16T03:11:19.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
225132025-12-16T03:11:19.328ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
225142025-12-16T03:11:19.328ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.114403ms iteration = 147 status = {"config_updated":false}
225152025-12-16T03:11:19.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
225162025-12-16T03:11:19.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225172025-12-16T03:11:19.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
225182025-12-16T03:11:19.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
225192025-12-16T03:11:19.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
225202025-12-16T03:11:19.348ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
225212025-12-16T03:11:19.348ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.846866ms iteration = 74 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
225222025-12-16T03:11:19.356ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
225232025-12-16T03:11:19.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
225242025-12-16T03:11:19.395ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 75.092881ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
225252025-12-16T03:11:21.070ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 269 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = c369829b-8a96-41ac-84ef-21b56b8be964 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
225262025-12-16T03:11:24.319ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 50 reason = Timeout
225272025-12-16T03:11:24.319ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 65 reason = Timeout
225282025-12-16T03:11:24.319ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
225292025-12-16T03:11:24.319ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
225302025-12-16T03:11:24.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
225312025-12-16T03:11:24.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
225322025-12-16T03:11:24.320ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 50 reason = Timeout
225332025-12-16T03:11:24.320ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 148 reason = Timeout
225342025-12-16T03:11:24.320ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
225352025-12-16T03:11:24.320ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
225362025-12-16T03:11:24.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
225372025-12-16T03:11:24.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
225382025-12-16T03:11:24.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
225392025-12-16T03:11:24.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
225402025-12-16T03:11:24.321ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225412025-12-16T03:11:24.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225422025-12-16T03:11:24.322ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225432025-12-16T03:11:24.322ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225442025-12-16T03:11:24.324ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
225452025-12-16T03:11:24.324ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
225462025-12-16T03:11:24.324ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
225472025-12-16T03:11:24.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
225482025-12-16T03:11:24.326ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 7.243292ms iteration = 50 status = "NoSitrep"
225492025-12-16T03:11:24.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
225502025-12-16T03:11:24.326ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
225512025-12-16T03:11:24.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
225522025-12-16T03:11:24.327ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
225532025-12-16T03:11:24.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
225542025-12-16T03:11:24.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
225552025-12-16T03:11:24.327ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
225562025-12-16T03:11:24.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
225572025-12-16T03:11:24.328ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
225582025-12-16T03:11:24.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
225592025-12-16T03:11:24.328ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 60390ee0-9ebf-417a-b74d-f8a1c4e08914 original_time_started = 2025-12-16 03:11:10.602552 UTC
225602025-12-16T03:11:24.328ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.866942ms iteration = 65 status = {"Loaded":{"collection_id":"60390ee0-9ebf-417a-b74d-f8a1c4e08914","time_loaded":"2025-12-16T03:11:24.319510655Z","time_started":"2025-12-16T03:11:10.602552Z"}}
225612025-12-16T03:11:24.329ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
225622025-12-16T03:11:24.329ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.888302ms iteration = 148 status = {"config_updated":false}
225632025-12-16T03:11:24.329ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225642025-12-16T03:11:24.330ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
225652025-12-16T03:11:24.331ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
225662025-12-16T03:11:24.331ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 11.394645ms iteration = 50 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
225672025-12-16T03:11:29.320ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
225682025-12-16T03:11:29.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
225692025-12-16T03:11:29.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
225702025-12-16T03:11:29.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
225712025-12-16T03:11:29.320ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 149 reason = Timeout
225722025-12-16T03:11:29.320ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
225732025-12-16T03:11:29.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
225742025-12-16T03:11:29.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
225752025-12-16T03:11:29.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
225762025-12-16T03:11:29.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225772025-12-16T03:11:29.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225782025-12-16T03:11:29.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
225792025-12-16T03:11:29.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
225802025-12-16T03:11:29.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
225812025-12-16T03:11:29.327ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
225822025-12-16T03:11:29.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
225832025-12-16T03:11:29.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
225842025-12-16T03:11:29.329ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
225852025-12-16T03:11:29.329ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.436737ms iteration = 149 status = {"config_updated":false}
225862025-12-16T03:11:29.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
225872025-12-16T03:11:29.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
225882025-12-16T03:11:29.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225892025-12-16T03:11:29.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
225902025-12-16T03:11:29.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
225912025-12-16T03:11:29.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
225922025-12-16T03:11:29.345ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
225932025-12-16T03:11:29.345ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.876325ms iteration = 75 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
225942025-12-16T03:11:29.362ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
225952025-12-16T03:11:29.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
225962025-12-16T03:11:29.364ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 43.40219ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
225972025-12-16T03:11:31.070ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 307 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 7531826d-54e5-46ba-bbbe-997c50ed6261 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
225982025-12-16T03:11:34.320ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 150 reason = Timeout
225992025-12-16T03:11:34.320ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
226002025-12-16T03:11:34.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
226012025-12-16T03:11:34.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226022025-12-16T03:11:34.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
226032025-12-16T03:11:34.326ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
226042025-12-16T03:11:34.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
226052025-12-16T03:11:34.328ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
226062025-12-16T03:11:34.328ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.005182ms iteration = 150 status = {"config_updated":false}
226072025-12-16T03:11:39.178ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
226082025-12-16T03:11:39.178ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 26 reason = Timeout
226092025-12-16T03:11:39.178ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
226102025-12-16T03:11:39.178ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
226112025-12-16T03:11:39.178ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
226122025-12-16T03:11:39.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
226132025-12-16T03:11:39.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
226142025-12-16T03:11:39.183ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1706, tv_nsec: 790306133 } })
226152025-12-16T03:11:39.184ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1706, tv_nsec: 791325796 } })
226162025-12-16T03:11:39.185ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
226172025-12-16T03:11:39.185ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1706, tv_nsec: 791905863 } })
226182025-12-16T03:11:39.187ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 8.436867ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
226192025-12-16T03:11:39.256ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
226202025-12-16T03:11:39.258ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 79.589309ms iteration = 26 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
226212025-12-16T03:11:39.265ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
226222025-12-16T03:11:39.265ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
226232025-12-16T03:11:39.265ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3564b706-4302-4331-a7cd-b623f9a64716", "content-length": "26", "date": "Tue, 16 Dec 2025 03:11:39 GMT"} })
226242025-12-16T03:11:39.265ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
226252025-12-16T03:11:39.267ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
226262025-12-16T03:11:39.267ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1706, tv_nsec: 874169986 } })
226272025-12-16T03:11:39.303ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 124.983894ms iteration = 26 status = {}
226282025-12-16T03:11:39.319ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
226292025-12-16T03:11:39.319ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 51 reason = Timeout
226302025-12-16T03:11:39.319ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 66 reason = Timeout
226312025-12-16T03:11:39.319ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
226322025-12-16T03:11:39.319ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
226332025-12-16T03:11:39.319ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
226342025-12-16T03:11:39.319ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 26 reason = Timeout
226352025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 17.23\u{b5}s iteration = 26 status = {"disabled":true,"errors":[],"sps":[]}
226362025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 39 reason = Timeout
226372025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
226382025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
226392025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
226402025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
226412025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
226422025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
226432025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
226442025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 26 reason = Timeout
226452025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
226462025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
226472025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
226482025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 51 reason = Timeout
226492025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
226502025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
226512025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
226522025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
226532025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
226542025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
226552025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 151 reason = Timeout
226562025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
226572025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
226582025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
226592025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
226602025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
226612025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
226622025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 39 reason = Timeout
226632025-12-16T03:11:39.320ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
226642025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
226652025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
226662025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
226672025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
226682025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
226692025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
226702025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 42 reason = Timeout
226712025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
226722025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
226732025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
226742025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
226752025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
226762025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
226772025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
226782025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
226792025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
226802025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
226812025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
226822025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
226832025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
226842025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
226852025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
226862025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
226872025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
226882025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
226892025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
226902025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
226912025-12-16T03:11:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
226922025-12-16T03:11:39.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
226932025-12-16T03:11:39.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
226942025-12-16T03:11:39.322ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226952025-12-16T03:11:39.322ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 2.594982ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
226962025-12-16T03:11:39.322ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226972025-12-16T03:11:39.322ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226982025-12-16T03:11:39.322ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226992025-12-16T03:11:39.322ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227002025-12-16T03:11:39.322ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227012025-12-16T03:11:39.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227022025-12-16T03:11:39.324ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
227032025-12-16T03:11:39.324ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227042025-12-16T03:11:39.324ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
227052025-12-16T03:11:39.324ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
227062025-12-16T03:11:39.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
227072025-12-16T03:11:39.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
227082025-12-16T03:11:39.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
227092025-12-16T03:11:39.326ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227102025-12-16T03:11:39.327ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 7.681288ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
227112025-12-16T03:11:39.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
227122025-12-16T03:11:39.328ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
227132025-12-16T03:11:39.328ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
227142025-12-16T03:11:39.328ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
227152025-12-16T03:11:39.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
227162025-12-16T03:11:39.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
227172025-12-16T03:11:39.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
227182025-12-16T03:11:39.329ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
227192025-12-16T03:11:39.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
227202025-12-16T03:11:39.329ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
227212025-12-16T03:11:39.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
227222025-12-16T03:11:39.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
227232025-12-16T03:11:39.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
227242025-12-16T03:11:39.330ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
227252025-12-16T03:11:39.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
227262025-12-16T03:11:39.330ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 10.899589ms iteration = 51 status = "NoSitrep"
227272025-12-16T03:11:39.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
227282025-12-16T03:11:39.330ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
227292025-12-16T03:11:39.331ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
227302025-12-16T03:11:39.331ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
227312025-12-16T03:11:39.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
227322025-12-16T03:11:39.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
227332025-12-16T03:11:39.331ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
227342025-12-16T03:11:39.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
227352025-12-16T03:11:39.331ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
227362025-12-16T03:11:39.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
227372025-12-16T03:11:39.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
227382025-12-16T03:11:39.332ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 60390ee0-9ebf-417a-b74d-f8a1c4e08914 original_time_started = 2025-12-16 03:11:10.602552 UTC
227392025-12-16T03:11:39.332ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 12.812773ms iteration = 66 status = {"Loaded":{"collection_id":"60390ee0-9ebf-417a-b74d-f8a1c4e08914","time_loaded":"2025-12-16T03:11:39.319927988Z","time_started":"2025-12-16T03:11:10.602552Z"}}
227402025-12-16T03:11:39.332ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
227412025-12-16T03:11:39.332ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 12.765422ms iteration = 151 status = {"config_updated":false}
227422025-12-16T03:11:39.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
227432025-12-16T03:11:39.333ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
227442025-12-16T03:11:39.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
227452025-12-16T03:11:39.333ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227462025-12-16T03:11:39.338ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
227472025-12-16T03:11:39.338ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
227482025-12-16T03:11:39.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
227492025-12-16T03:11:39.338ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1706, tv_nsec: 945165545 } })
227502025-12-16T03:11:39.338ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
227512025-12-16T03:11:39.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
227522025-12-16T03:11:39.339ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
227532025-12-16T03:11:39.339ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1706, tv_nsec: 946291579 } })
227542025-12-16T03:11:39.342ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 22.368593ms iteration = 26 status = 0
227552025-12-16T03:11:39.343ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227562025-12-16T03:11:39.347ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
227572025-12-16T03:11:39.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
227582025-12-16T03:11:39.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
227592025-12-16T03:11:39.363ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
227602025-12-16T03:11:39.363ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 43.229938ms iteration = 76 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
227612025-12-16T03:11:39.374ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 54.264948ms iteration = 26 status = {"errors":[],"finish_invoked_ok":[]}
227622025-12-16T03:11:39.374ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 54.821235ms iteration = 26 status = {"errors":[],"requests_created_ok":[]}
227632025-12-16T03:11:39.375ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
227642025-12-16T03:11:39.375ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
227652025-12-16T03:11:39.376ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7f6055f-edc7-4a42-83b0-08baae516063", "content-length": "26", "date": "Tue, 16 Dec 2025 03:11:39 GMT"} })
227662025-12-16T03:11:39.376ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
227672025-12-16T03:11:39.376ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 56.509686ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
227682025-12-16T03:11:39.377ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
227692025-12-16T03:11:39.379ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
227702025-12-16T03:11:39.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
227712025-12-16T03:11:39.380ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227722025-12-16T03:11:39.381ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
227732025-12-16T03:11:39.383ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
227742025-12-16T03:11:39.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
227752025-12-16T03:11:39.384ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
227762025-12-16T03:11:39.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
227772025-12-16T03:11:39.385ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
227782025-12-16T03:11:39.386ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
227792025-12-16T03:11:39.386ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227802025-12-16T03:11:39.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
227812025-12-16T03:11:39.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
227822025-12-16T03:11:39.388ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 68.337226ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
227832025-12-16T03:11:39.388ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227842025-12-16T03:11:39.388ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
227852025-12-16T03:11:39.388ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
227862025-12-16T03:11:39.388ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
227872025-12-16T03:11:39.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
227882025-12-16T03:11:39.389ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
227892025-12-16T03:11:39.389ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
227902025-12-16T03:11:39.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
227912025-12-16T03:11:39.390ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
227922025-12-16T03:11:39.390ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
227932025-12-16T03:11:39.390ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 70.875279ms iteration = 51 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
227942025-12-16T03:11:39.391ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 70.914448ms iteration = 42 status = {"physical_disks_added":0}
227952025-12-16T03:11:39.391ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227962025-12-16T03:11:39.392ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
227972025-12-16T03:11:39.392ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
227982025-12-16T03:11:39.392ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
227992025-12-16T03:11:39.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
228002025-12-16T03:11:39.394ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
228012025-12-16T03:11:39.394ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
228022025-12-16T03:11:39.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
228032025-12-16T03:11:39.395ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
228042025-12-16T03:11:39.395ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 75.224833ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
228052025-12-16T03:11:39.398ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
228062025-12-16T03:11:39.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
228072025-12-16T03:11:39.400ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
228082025-12-16T03:11:39.402ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
228092025-12-16T03:11:39.402ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
228102025-12-16T03:11:39.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
228112025-12-16T03:11:39.404ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
228122025-12-16T03:11:39.404ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 84.38905ms iteration = 39 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
228132025-12-16T03:11:39.420ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1707, tv_nsec: 27529939 } })
228142025-12-16T03:11:39.424ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 104.14784ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
228152025-12-16T03:11:39.439ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
228162025-12-16T03:11:39.439ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
228172025-12-16T03:11:39.439ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 119.755427ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
228182025-12-16T03:11:39.453ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
228192025-12-16T03:11:39.453ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
228202025-12-16T03:11:39.458ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8dcdb3d3-ab7f-4fb8-8caa-a325eeea0f9e", "content-length": "372", "date": "Tue, 16 Dec 2025 03:11:39 GMT"} })
228212025-12-16T03:11:39.458ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
228222025-12-16T03:11:39.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
228232025-12-16T03:11:39.467ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1707, tv_nsec: 74456063 } })
228242025-12-16T03:11:39.471ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
228252025-12-16T03:11:39.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
228262025-12-16T03:11:39.474ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
228272025-12-16T03:11:39.474ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
228282025-12-16T03:11:39.475ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "06a57345-2a67-4a1a-8d8f-4ef59e08bf7f", "date": "Tue, 16 Dec 2025 03:11:39 GMT"} })
228292025-12-16T03:11:39.475ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
228302025-12-16T03:11:39.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
228312025-12-16T03:11:39.478ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
228322025-12-16T03:11:39.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
228332025-12-16T03:11:39.481ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
228342025-12-16T03:11:39.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
228352025-12-16T03:11:39.484ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
228362025-12-16T03:11:39.484ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
228372025-12-16T03:11:39.489ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "f96512b7-5dc0-4e2d-b300-0998cbd7cb43", "date": "Tue, 16 Dec 2025 03:11:39 GMT"} })
228382025-12-16T03:11:39.491ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 171.275939ms iteration = 39 status = {}
228392025-12-16T03:11:39.498ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
228402025-12-16T03:11:39.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
228412025-12-16T03:11:39.501ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
228422025-12-16T03:11:39.501ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
228432025-12-16T03:11:39.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
228442025-12-16T03:11:39.512ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
228452025-12-16T03:11:39.512ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
228462025-12-16T03:11:39.513ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
228472025-12-16T03:11:39.513ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
228482025-12-16T03:11:39.513ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
228492025-12-16T03:11:39.513ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
228502025-12-16T03:11:39.513ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
228512025-12-16T03:11:39.513ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
228522025-12-16T03:11:39.513ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
228532025-12-16T03:11:39.513ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
228542025-12-16T03:11:39.514ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
228552025-12-16T03:11:39.514ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
228562025-12-16T03:11:39.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
228572025-12-16T03:11:39.516ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
228582025-12-16T03:11:39.516ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
228592025-12-16T03:11:39.516ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
228602025-12-16T03:11:39.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
228612025-12-16T03:11:39.517ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
228622025-12-16T03:11:39.520ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
228632025-12-16T03:11:39.520ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
228642025-12-16T03:11:39.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
228652025-12-16T03:11:39.523ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
228662025-12-16T03:11:39.528ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
228672025-12-16T03:11:39.528ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
228682025-12-16T03:11:39.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
228692025-12-16T03:11:39.562ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
228702025-12-16T03:11:39.563ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "fcebd991-f1b5-43fd-ae2a-6d7159432c0f", "content-length": "691", "date": "Tue, 16 Dec 2025 03:11:39 GMT"} })
228712025-12-16T03:11:39.563ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
228722025-12-16T03:11:39.563ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
228732025-12-16T03:11:39.563ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
228742025-12-16T03:11:39.565ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 245.294107ms iteration = 26 status = {}
228752025-12-16T03:11:39.675ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "7dbbbe62-99ce-41aa-8cfb-63986dce2fd9", "date": "Tue, 16 Dec 2025 03:11:39 GMT"} })
228762025-12-16T03:11:39.675ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
228772025-12-16T03:11:39.675ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
228782025-12-16T03:11:39.675ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a6e753d-034a-4f5c-8af0-a263e9144b95", "content-length": "682", "date": "Tue, 16 Dec 2025 03:11:39 GMT"} })
228792025-12-16T03:11:39.676ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
228802025-12-16T03:11:39.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
228812025-12-16T03:11:39.679ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
228822025-12-16T03:11:39.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
228832025-12-16T03:11:39.682ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
228842025-12-16T03:11:39.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
228852025-12-16T03:11:39.685ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
228862025-12-16T03:11:39.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
228872025-12-16T03:11:39.693ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 373.147718ms iteration = 26 status = {}
228882025-12-16T03:11:41.070ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 468 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 9c2b6ac1-296d-42ab-a3ce-8d9f915f38f8 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
228892025-12-16T03:11:44.320ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 152 reason = Timeout
228902025-12-16T03:11:44.320ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
228912025-12-16T03:11:44.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
228922025-12-16T03:11:44.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
228932025-12-16T03:11:44.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
228942025-12-16T03:11:44.326ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
228952025-12-16T03:11:44.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
228962025-12-16T03:11:44.329ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
228972025-12-16T03:11:44.329ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.196204ms iteration = 152 status = {"config_updated":false}
228982025-12-16T03:11:49.320ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
228992025-12-16T03:11:49.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
229002025-12-16T03:11:49.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
229012025-12-16T03:11:49.320ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
229022025-12-16T03:11:49.320ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 153 reason = Timeout
229032025-12-16T03:11:49.320ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
229042025-12-16T03:11:49.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
229052025-12-16T03:11:49.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
229062025-12-16T03:11:49.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
229072025-12-16T03:11:49.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229082025-12-16T03:11:49.322ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
229092025-12-16T03:11:49.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
229102025-12-16T03:11:49.322ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229112025-12-16T03:11:49.324ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.401066ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
229122025-12-16T03:11:49.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
229132025-12-16T03:11:49.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
229142025-12-16T03:11:49.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
229152025-12-16T03:11:49.327ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
229162025-12-16T03:11:49.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
229172025-12-16T03:11:49.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
229182025-12-16T03:11:49.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
229192025-12-16T03:11:49.328ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
229202025-12-16T03:11:49.328ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.219154ms iteration = 153 status = {"config_updated":false}
229212025-12-16T03:11:49.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
229222025-12-16T03:11:49.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229232025-12-16T03:11:49.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
229242025-12-16T03:11:49.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
229252025-12-16T03:11:49.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
229262025-12-16T03:11:49.349ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
229272025-12-16T03:11:49.350ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.755047ms iteration = 77 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
229282025-12-16T03:11:51.071ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 286 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 2f6d4549-da28-4b8a-93e2-bb6f881ee64b response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
229292025-12-16T03:11:54.319ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 52 reason = Timeout
229302025-12-16T03:11:54.319ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
229312025-12-16T03:11:54.319ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 67 reason = Timeout
229322025-12-16T03:11:54.319ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
229332025-12-16T03:11:54.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
229342025-12-16T03:11:54.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
229352025-12-16T03:11:54.320ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 52 reason = Timeout
229362025-12-16T03:11:54.320ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 154 reason = Timeout
229372025-12-16T03:11:54.320ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
229382025-12-16T03:11:54.320ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
229392025-12-16T03:11:54.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
229402025-12-16T03:11:54.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
229412025-12-16T03:11:54.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
229422025-12-16T03:11:54.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
229432025-12-16T03:11:54.321ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229442025-12-16T03:11:54.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229452025-12-16T03:11:54.322ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229462025-12-16T03:11:54.323ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229472025-12-16T03:11:54.325ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
229482025-12-16T03:11:54.325ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
229492025-12-16T03:11:54.325ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
229502025-12-16T03:11:54.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
229512025-12-16T03:11:54.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
229522025-12-16T03:11:54.326ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
229532025-12-16T03:11:54.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
229542025-12-16T03:11:54.326ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 7.649877ms iteration = 52 status = "NoSitrep"
229552025-12-16T03:11:54.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
229562025-12-16T03:11:54.327ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
229572025-12-16T03:11:54.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
229582025-12-16T03:11:54.328ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
229592025-12-16T03:11:54.328ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.752578ms iteration = 154 status = {"config_updated":false}
229602025-12-16T03:11:54.328ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 60390ee0-9ebf-417a-b74d-f8a1c4e08914 original_time_started = 2025-12-16 03:11:10.602552 UTC
229612025-12-16T03:11:54.328ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 9.138816ms iteration = 67 status = {"Loaded":{"collection_id":"60390ee0-9ebf-417a-b74d-f8a1c4e08914","time_loaded":"2025-12-16T03:11:54.319342267Z","time_started":"2025-12-16T03:11:10.602552Z"}}
229622025-12-16T03:11:54.328ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
229632025-12-16T03:11:54.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
229642025-12-16T03:11:54.329ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
229652025-12-16T03:11:54.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
229662025-12-16T03:11:54.331ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229672025-12-16T03:11:54.332ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
229682025-12-16T03:11:54.333ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
229692025-12-16T03:11:54.333ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 12.449628ms iteration = 52 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
229702025-12-16T03:11:59.320ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
229712025-12-16T03:11:59.321ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
229722025-12-16T03:11:59.321ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
229732025-12-16T03:11:59.321ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 155 reason = Timeout
229742025-12-16T03:11:59.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
229752025-12-16T03:11:59.321ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
229762025-12-16T03:11:59.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
229772025-12-16T03:11:59.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
229782025-12-16T03:11:59.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
229792025-12-16T03:11:59.322ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
229802025-12-16T03:11:59.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229812025-12-16T03:11:59.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229822025-12-16T03:11:59.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
229832025-12-16T03:11:59.324ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.510655ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
229842025-12-16T03:11:59.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
229852025-12-16T03:11:59.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
229862025-12-16T03:11:59.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
229872025-12-16T03:11:59.327ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
229882025-12-16T03:11:59.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
229892025-12-16T03:11:59.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
229902025-12-16T03:11:59.328ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
229912025-12-16T03:11:59.328ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.674967ms iteration = 155 status = {"config_updated":false}
229922025-12-16T03:11:59.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
229932025-12-16T03:11:59.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
229942025-12-16T03:11:59.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229952025-12-16T03:11:59.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
229962025-12-16T03:11:59.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
229972025-12-16T03:11:59.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
229982025-12-16T03:11:59.344ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
229992025-12-16T03:11:59.344ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.73383ms iteration = 78 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
230002025-12-16T03:12:01.070ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 303 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = f936dc32-986b-4c04-ae49-d12a394b89b4 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
230012025-12-16T03:12:04.321ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 156 reason = Timeout
230022025-12-16T03:12:04.321ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
230032025-12-16T03:12:04.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
230042025-12-16T03:12:04.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
230052025-12-16T03:12:04.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
230062025-12-16T03:12:04.326ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
230072025-12-16T03:12:04.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
230082025-12-16T03:12:04.328ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
230092025-12-16T03:12:04.328ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.316403ms iteration = 156 status = {"config_updated":false}
230102025-12-16T03:12:09.178ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 14 reason = Timeout
230112025-12-16T03:12:09.178ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
230122025-12-16T03:12:09.178ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 14 reason = Timeout
230132025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
230142025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
230152025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 14 reason = Timeout
230162025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
230172025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
230182025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 14 reason = Timeout
230192025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 14 reason = Timeout
230202025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
230212025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
230222025-12-16T03:12:09.179ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-12-16 03:02:09.179022486 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
230232025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 27 reason = Timeout
230242025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
230252025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
230262025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
230272025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
230282025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
230292025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
230302025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
230312025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
230322025-12-16T03:12:09.179ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
230332025-12-16T03:12:09.179ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2994091c-bbd4-4473-ae42-b9bc8dd03211 step component = ExternalNetworking step id = Ensure
230342025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
230352025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
230362025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
230372025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
230382025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
230392025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
230402025-12-16T03:12:09.179ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("1a45cd23-3d0a-43eb-adce-62c7b712baf1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("8addbd1b-2c59-4c49-96f1-40684614ace0.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::8)) }], valid_until: Instant { tv_sec: 1736, tv_nsec: 785886607 } })
230412025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
230422025-12-16T03:12:09.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
230432025-12-16T03:12:09.179ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("49a51a59-9919-48ae-8182-4070a8430160.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("6af60386-8771-4401-9d4d-e93774e519f0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("87dd4f81-0d39-4f7a-962b-d01d4c876a84.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 1736, tv_nsec: 786004398 } })
230442025-12-16T03:12:09.180ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
230452025-12-16T03:12:09.180ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } dns_group = external
230462025-12-16T03:12:09.180ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.680072ms iteration = 14 status = {"addresses":["[fd00:1122:3344:101::9]:5353","[fd00:1122:3344:101::8]:5353"]}
230472025-12-16T03:12:09.180ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.5551ms iteration = 14 status = {"addresses":["[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:1::1]:5353"]}
230482025-12-16T03:12:09.180ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
230492025-12-16T03:12:09.181ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
230502025-12-16T03:12:09.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
230512025-12-16T03:12:09.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
230522025-12-16T03:12:09.184ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
230532025-12-16T03:12:09.185ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
230542025-12-16T03:12:09.185ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1736, tv_nsec: 791838402 } })
230552025-12-16T03:12:09.186ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
230562025-12-16T03:12:09.187ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-12-16 03:02:09.179022486 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
230572025-12-16T03:12:09.187ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 8.108293ms iteration = 14 status = {"expiration":"2025-12-16T03:02:09.179022486Z","pruned":[]}
230582025-12-16T03:12:09.188ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1736, tv_nsec: 794230693 } })
230592025-12-16T03:12:09.188ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
230602025-12-16T03:12:09.188ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1736, tv_nsec: 795017063 } })
230612025-12-16T03:12:09.189ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
230622025-12-16T03:12:09.189ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
230632025-12-16T03:12:09.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
230642025-12-16T03:12:09.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
230652025-12-16T03:12:09.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
230662025-12-16T03:12:09.193ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
230672025-12-16T03:12:09.193ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
230682025-12-16T03:12:09.194ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 15.314084ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
230692025-12-16T03:12:09.195ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
230702025-12-16T03:12:09.195ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
230712025-12-16T03:12:09.195ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
230722025-12-16T03:12:09.195ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
230732025-12-16T03:12:09.195ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
230742025-12-16T03:12:09.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
230752025-12-16T03:12:09.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
230762025-12-16T03:12:09.195ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
230772025-12-16T03:12:09.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
230782025-12-16T03:12:09.196ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
230792025-12-16T03:12:09.196ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
230802025-12-16T03:12:09.196ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
230812025-12-16T03:12:09.196ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
230822025-12-16T03:12:09.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
230832025-12-16T03:12:09.197ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
230842025-12-16T03:12:09.199ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
230852025-12-16T03:12:09.203ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
230862025-12-16T03:12:09.204ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
230872025-12-16T03:12:09.204ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
230882025-12-16T03:12:09.204ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
230892025-12-16T03:12:09.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
230902025-12-16T03:12:09.205ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
230912025-12-16T03:12:09.207ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
230922025-12-16T03:12:09.207ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
230932025-12-16T03:12:09.207ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
230942025-12-16T03:12:09.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
230952025-12-16T03:12:09.210ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
230962025-12-16T03:12:09.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
230972025-12-16T03:12:09.210ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
230982025-12-16T03:12:09.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
230992025-12-16T03:12:09.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
231002025-12-16T03:12:09.211ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
231012025-12-16T03:12:09.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
231022025-12-16T03:12:09.212ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
231032025-12-16T03:12:09.213ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
231042025-12-16T03:12:09.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
231052025-12-16T03:12:09.215ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
231062025-12-16T03:12:09.216ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
231072025-12-16T03:12:09.216ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
231082025-12-16T03:12:09.216ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
231092025-12-16T03:12:09.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
231102025-12-16T03:12:09.217ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: e5dac41c-2c35-432c-8898-8110926454ec (external_ip), ip: 10.151.1.168 }) nic = NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1511ffa1-5095-440c-978b-25339cc8be11 zone_kind = nexus
231112025-12-16T03:12:09.219ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
231122025-12-16T03:12:09.219ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
231132025-12-16T03:12:09.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
231142025-12-16T03:12:09.219ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
231152025-12-16T03:12:09.219ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
231162025-12-16T03:12:09.219ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
231172025-12-16T03:12:09.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
231182025-12-16T03:12:09.220ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 32 reason = Timeout
231192025-12-16T03:12:09.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
231202025-12-16T03:12:09.220ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
231212025-12-16T03:12:09.220ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
231222025-12-16T03:12:09.220ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
231232025-12-16T03:12:09.220ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
231242025-12-16T03:12:09.220ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
231252025-12-16T03:12:09.220ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
231262025-12-16T03:12:09.220ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
231272025-12-16T03:12:09.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
231282025-12-16T03:12:09.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
231292025-12-16T03:12:09.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
231302025-12-16T03:12:09.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
231312025-12-16T03:12:09.224ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
231322025-12-16T03:12:09.224ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
231332025-12-16T03:12:09.224ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
231342025-12-16T03:12:09.224ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
231352025-12-16T03:12:09.225ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-12-16 02:59:16.791641 UTC dns_group = internal generation = 1
231362025-12-16T03:12:09.225ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 46.073114ms iteration = 14 status = {"generation":1}
231372025-12-16T03:12:09.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
231382025-12-16T03:12:09.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
231392025-12-16T03:12:09.225ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
231402025-12-16T03:12:09.225ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
231412025-12-16T03:12:09.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
231422025-12-16T03:12:09.227ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
231432025-12-16T03:12:09.229ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
231442025-12-16T03:12:09.229ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
231452025-12-16T03:12:09.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
231462025-12-16T03:12:09.229ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 08ae9a4f-e832-4313-af01-2240814fb695 (external_ip), ip: 10.151.1.165 }) nic = NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 } zone_id = 1a45cd23-3d0a-43eb-adce-62c7b712baf1 zone_kind = external_dns
231472025-12-16T03:12:09.231ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
231482025-12-16T03:12:09.231ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
231492025-12-16T03:12:09.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
231502025-12-16T03:12:09.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
231512025-12-16T03:12:09.236ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
231522025-12-16T03:12:09.236ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
231532025-12-16T03:12:09.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
231542025-12-16T03:12:09.237ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 45 reason = Timeout
231552025-12-16T03:12:09.237ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
231562025-12-16T03:12:09.238ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
231572025-12-16T03:12:09.238ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
231582025-12-16T03:12:09.238ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 14 reason = Timeout
231592025-12-16T03:12:09.238ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 33.9\u{b5}s iteration = 45 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
231602025-12-16T03:12:09.238ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
231612025-12-16T03:12:09.238ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 14 reason = Timeout
231622025-12-16T03:12:09.238ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
231632025-12-16T03:12:09.238ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 14 reason = Timeout
231642025-12-16T03:12:09.238ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
231652025-12-16T03:12:09.238ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
231662025-12-16T03:12:09.238ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
231672025-12-16T03:12:09.238ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
231682025-12-16T03:12:09.238ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-12-16 02:59:18.154440 UTC dns_group = external generation = 2
231692025-12-16T03:12:09.238ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 59.20511ms iteration = 14 status = {"generation":2}
231702025-12-16T03:12:09.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
231712025-12-16T03:12:09.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
231722025-12-16T03:12:09.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
231732025-12-16T03:12:09.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
231742025-12-16T03:12:09.240ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
231752025-12-16T03:12:09.241ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
231762025-12-16T03:12:09.242ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
231772025-12-16T03:12:09.242ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 4.046242ms iteration = 14 status = {"by_rx":{},"error":null}
231782025-12-16T03:12:09.242ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.521907ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
231792025-12-16T03:12:09.244ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
231802025-12-16T03:12:09.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
231812025-12-16T03:12:09.245ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
231822025-12-16T03:12:09.245ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
231832025-12-16T03:12:09.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
231842025-12-16T03:12:09.247ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Snat(OmicronZoneExternalSnatIp { id: b8ee4538-777d-4d66-9509-bff79852455f (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: e949bd68-45f7-471c-a2ae-8e03a4c7b63a, kind: Service { id: 5f72890f-30aa-4ce1-8d39-5b7bd438f00c }, name: Name("ntp-5f72890f-30aa-4ce1-8d39-5b7bd438f00c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.3.5, subnet: Ipv4Net { addr: 172.30.3.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 165, 117])), vni: Vni(100), primary: true, slot: 0 } zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c zone_kind = boundary_ntp
231852025-12-16T03:12:09.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
231862025-12-16T03:12:09.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
231872025-12-16T03:12:09.248ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 10.923438ms iteration = 14 status = {"dispatched":[],"errors":[],"glob_version":"212.0.0","globs_reprocessed":{},"no_receivers":[]}
231882025-12-16T03:12:09.249ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
231892025-12-16T03:12:09.249ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
231902025-12-16T03:12:09.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
231912025-12-16T03:12:09.255ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
231922025-12-16T03:12:09.255ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
231932025-12-16T03:12:09.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
231942025-12-16T03:12:09.256ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 33c3cc40-6f12-44c3-8329-025ccb28f09f (external_ip), ip: 10.151.1.166 }) nic = NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 } zone_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c zone_kind = nexus
231952025-12-16T03:12:09.256ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
231962025-12-16T03:12:09.256ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
231972025-12-16T03:12:09.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
231982025-12-16T03:12:09.267ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
231992025-12-16T03:12:09.271ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: dd286ca2-1548-4cda-b5ae-3feef4bebd5f (external_ip), ip: 10.151.1.164 }) nic = NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8addbd1b-2c59-4c49-96f1-40684614ace0 zone_kind = external_dns
232002025-12-16T03:12:09.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232012025-12-16T03:12:09.274ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:253 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
232022025-12-16T03:12:09.275ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
232032025-12-16T03:12:09.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
232042025-12-16T03:12:09.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
232052025-12-16T03:12:09.276ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:320 ip = Floating(OmicronZoneExternalFloatingIp { id: 5204577e-c66a-41c0-87b2-25f084293040 (external_ip), ip: 10.151.1.167 }) nic = NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 } zone_id = 8f641c55-8873-4292-9e77-436129b8e195 zone_kind = nexus
232062025-12-16T03:12:09.277ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2994091c-bbd4-4473-ae42-b9bc8dd03211 step component = SledList step id = Fetch
232072025-12-16T03:12:09.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
232082025-12-16T03:12:09.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
232092025-12-16T03:12:09.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232102025-12-16T03:12:09.287ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
232112025-12-16T03:12:09.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
232122025-12-16T03:12:09.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
232132025-12-16T03:12:09.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232142025-12-16T03:12:09.294ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
232152025-12-16T03:12:09.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
232162025-12-16T03:12:09.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
232172025-12-16T03:12:09.309ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
232182025-12-16T03:12:09.309ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
232192025-12-16T03:12:09.309ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 89.13156ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
232202025-12-16T03:12:09.320ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-12-16T02:59:05.020237Z, time_modified: 2025-12-16T02:59:05.020237Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, db_silo: Silo { identity: SiloIdentity { id: eac5ed97-074a-4275-ab9e-630b9a7a8ca7, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-12-16T02:59:17.964454Z, time_modified: 2025-12-16T02:59:17.964454Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29" }] }) }
232212025-12-16T03:12:09.320ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 141.090287ms iteration = 14 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]}},"default_endpoint":{"silo_id":"eac5ed97-074a-4275-ab9e-630b9a7a8ca7","tls_certs":["6d0fd334ee6f4ff585de21e5e5cb20d6dafc6285b9aad425b80429f69b9bde29"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
232222025-12-16T03:12:09.320ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
232232025-12-16T03:12:09.320ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
232242025-12-16T03:12:09.320ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
232252025-12-16T03:12:09.320ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 53 reason = Timeout
232262025-12-16T03:12:09.320ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
232272025-12-16T03:12:09.320ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
232282025-12-16T03:12:09.320ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 68 reason = Timeout
232292025-12-16T03:12:09.320ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
232302025-12-16T03:12:09.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
232312025-12-16T03:12:09.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
232322025-12-16T03:12:09.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
232332025-12-16T03:12:09.321ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
232342025-12-16T03:12:09.321ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
232352025-12-16T03:12:09.321ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 40 reason = Timeout
232362025-12-16T03:12:09.321ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
232372025-12-16T03:12:09.321ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
232382025-12-16T03:12:09.321ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
232392025-12-16T03:12:09.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
232402025-12-16T03:12:09.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
232412025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 27 reason = Timeout
232422025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
232432025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 53 reason = Timeout
232442025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
232452025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
232462025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
232472025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
232482025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
232492025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
232502025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
232512025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
232522025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
232532025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 157 reason = Timeout
232542025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
232552025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
232562025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
232572025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
232582025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
232592025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
232602025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
232612025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
232622025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
232632025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 40 reason = Timeout
232642025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
232652025-12-16T03:12:09.322ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
232662025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 27 reason = Timeout
232672025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
232682025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 12.47\u{b5}s iteration = 27 status = {"disabled":true,"errors":[],"sps":[]}
232692025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
232702025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 43 reason = Timeout
232712025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
232722025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
232732025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
232742025-12-16T03:12:09.322ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ea5040f-f168-4440-9d7d-c2edf088f3ec", "content-length": "26", "date": "Tue, 16 Dec 2025 03:12:09 GMT"} })
232752025-12-16T03:12:09.322ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
232762025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
232772025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
232782025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
232792025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
232802025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
232812025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
232822025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
232832025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
232842025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
232852025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
232862025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
232872025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
232882025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
232892025-12-16T03:12:09.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
232902025-12-16T03:12:09.323ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
232912025-12-16T03:12:09.324ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232922025-12-16T03:12:09.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
232932025-12-16T03:12:09.325ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232942025-12-16T03:12:09.325ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232952025-12-16T03:12:09.326ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232962025-12-16T03:12:09.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232972025-12-16T03:12:09.326ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232982025-12-16T03:12:09.328ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
232992025-12-16T03:12:09.328ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
233002025-12-16T03:12:09.328ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 108.055869ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
233012025-12-16T03:12:09.329ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
233022025-12-16T03:12:09.330ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
233032025-12-16T03:12:09.330ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
233042025-12-16T03:12:09.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
233052025-12-16T03:12:09.330ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
233062025-12-16T03:12:09.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
233072025-12-16T03:12:09.331ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
233082025-12-16T03:12:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
233092025-12-16T03:12:09.332ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
233102025-12-16T03:12:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
233112025-12-16T03:12:09.332ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
233122025-12-16T03:12:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
233132025-12-16T03:12:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
233142025-12-16T03:12:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
233152025-12-16T03:12:09.332ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
233162025-12-16T03:12:09.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
233172025-12-16T03:12:09.333ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 13.532832ms iteration = 53 status = "NoSitrep"
233182025-12-16T03:12:09.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
233192025-12-16T03:12:09.334ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
233202025-12-16T03:12:09.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
233212025-12-16T03:12:09.335ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
233222025-12-16T03:12:09.335ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
233232025-12-16T03:12:09.335ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
233242025-12-16T03:12:09.335ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
233252025-12-16T03:12:09.335ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
233262025-12-16T03:12:09.335ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
233272025-12-16T03:12:09.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
233282025-12-16T03:12:09.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
233292025-12-16T03:12:09.336ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
233302025-12-16T03:12:09.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
233312025-12-16T03:12:09.338ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1736, tv_nsec: 944948492 } })
233322025-12-16T03:12:09.339ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
233332025-12-16T03:12:09.339ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
233342025-12-16T03:12:09.340ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1736, tv_nsec: 946139568 } })
233352025-12-16T03:12:09.340ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1736, tv_nsec: 946199428 } })
233362025-12-16T03:12:09.341ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
233372025-12-16T03:12:09.341ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
233382025-12-16T03:12:09.341ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
233392025-12-16T03:12:09.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
233402025-12-16T03:12:09.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
233412025-12-16T03:12:09.345ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 60390ee0-9ebf-417a-b74d-f8a1c4e08914 original_time_started = 2025-12-16 03:11:10.602552 UTC
233422025-12-16T03:12:09.345ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 24.891256ms iteration = 68 status = {"Loaded":{"collection_id":"60390ee0-9ebf-417a-b74d-f8a1c4e08914","time_loaded":"2025-12-16T03:12:09.320312616Z","time_started":"2025-12-16T03:11:10.602552Z"}}
233432025-12-16T03:12:09.345ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
233442025-12-16T03:12:09.345ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 24.050205ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
233452025-12-16T03:12:09.346ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
233462025-12-16T03:12:09.346ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
233472025-12-16T03:12:09.348ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
233482025-12-16T03:12:09.349ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
233492025-12-16T03:12:09.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
233502025-12-16T03:12:09.350ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 28.814115ms iteration = 27 status = 0
233512025-12-16T03:12:09.352ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
233522025-12-16T03:12:09.352ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
233532025-12-16T03:12:09.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
233542025-12-16T03:12:09.356ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
233552025-12-16T03:12:09.362ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
233562025-12-16T03:12:09.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
233572025-12-16T03:12:09.365ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
233582025-12-16T03:12:09.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
233592025-12-16T03:12:09.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
233602025-12-16T03:12:09.365ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
233612025-12-16T03:12:09.367ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
233622025-12-16T03:12:09.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
233632025-12-16T03:12:09.368ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
233642025-12-16T03:12:09.370ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
233652025-12-16T03:12:09.370ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 48.315102ms iteration = 53 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
233662025-12-16T03:12:09.390ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
233672025-12-16T03:12:09.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
233682025-12-16T03:12:09.395ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 72.814472ms iteration = 43 status = {"physical_disks_added":0}
233692025-12-16T03:12:09.396ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
233702025-12-16T03:12:09.396ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 74.12272ms iteration = 79 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
233712025-12-16T03:12:09.396ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
233722025-12-16T03:12:09.410ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
233732025-12-16T03:12:09.411ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
233742025-12-16T03:12:09.413ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
233752025-12-16T03:12:09.415ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
233762025-12-16T03:12:09.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
233772025-12-16T03:12:09.416ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
233782025-12-16T03:12:09.416ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
233792025-12-16T03:12:09.416ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
233802025-12-16T03:12:09.417ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
233812025-12-16T03:12:09.417ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
233822025-12-16T03:12:09.417ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 238.387491ms iteration = 27 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
233832025-12-16T03:12:09.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
233842025-12-16T03:12:09.418ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
233852025-12-16T03:12:09.420ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 97.964701ms iteration = 27 status = {"errors":[],"finish_invoked_ok":[]}
233862025-12-16T03:12:09.420ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 98.010432ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
233872025-12-16T03:12:09.420ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
233882025-12-16T03:12:09.420ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 98.032193ms iteration = 157 status = {"config_updated":false}
233892025-12-16T03:12:09.421ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 99.44901ms iteration = 27 status = {"errors":[],"requests_created_ok":[]}
233902025-12-16T03:12:09.421ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
233912025-12-16T03:12:09.422ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
233922025-12-16T03:12:09.422ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
233932025-12-16T03:12:09.422ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
233942025-12-16T03:12:09.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
233952025-12-16T03:12:09.422ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
233962025-12-16T03:12:09.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
233972025-12-16T03:12:09.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
233982025-12-16T03:12:09.428ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 106.327087ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
233992025-12-16T03:12:09.430ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
234002025-12-16T03:12:09.430ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
234012025-12-16T03:12:09.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
234022025-12-16T03:12:09.430ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
234032025-12-16T03:12:09.431ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
234042025-12-16T03:12:09.432ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 110.812064ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
234052025-12-16T03:12:09.438ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 116.04654ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
234062025-12-16T03:12:09.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
234072025-12-16T03:12:09.444ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234082025-12-16T03:12:09.445ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
234092025-12-16T03:12:09.445ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
234102025-12-16T03:12:09.445ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
234112025-12-16T03:12:09.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
234122025-12-16T03:12:09.448ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
234132025-12-16T03:12:09.448ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
234142025-12-16T03:12:09.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
234152025-12-16T03:12:09.449ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
234162025-12-16T03:12:09.450ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
234172025-12-16T03:12:09.450ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
234182025-12-16T03:12:09.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
234192025-12-16T03:12:09.480ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234202025-12-16T03:12:09.484ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
234212025-12-16T03:12:09.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234222025-12-16T03:12:09.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234232025-12-16T03:12:09.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234242025-12-16T03:12:09.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234252025-12-16T03:12:09.489ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
234262025-12-16T03:12:09.489ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
234272025-12-16T03:12:09.490ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "dbdba34a-6a19-463f-a776-d87c725785f6", "content-length": "26", "date": "Tue, 16 Dec 2025 03:12:09 GMT"} })
234282025-12-16T03:12:09.490ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
234292025-12-16T03:12:09.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234302025-12-16T03:12:09.493ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
234312025-12-16T03:12:09.493ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
234322025-12-16T03:12:09.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
234332025-12-16T03:12:09.495ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 173.74035ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
234342025-12-16T03:12:09.496ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
234352025-12-16T03:12:09.496ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234362025-12-16T03:12:09.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234372025-12-16T03:12:09.498ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234382025-12-16T03:12:09.499ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234392025-12-16T03:12:09.501ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2994091c-bbd4-4473-ae42-b9bc8dd03211 step component = DeployNexusRecords step id = Ensure
234402025-12-16T03:12:09.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
234412025-12-16T03:12:09.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
234422025-12-16T03:12:09.503ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
234432025-12-16T03:12:09.503ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
234442025-12-16T03:12:09.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
234452025-12-16T03:12:09.504ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
234462025-12-16T03:12:09.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234472025-12-16T03:12:09.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234482025-12-16T03:12:09.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234492025-12-16T03:12:09.508ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
234502025-12-16T03:12:09.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
234512025-12-16T03:12:09.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
234522025-12-16T03:12:09.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234532025-12-16T03:12:09.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234542025-12-16T03:12:09.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
234552025-12-16T03:12:09.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
234562025-12-16T03:12:09.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234572025-12-16T03:12:09.519ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
234582025-12-16T03:12:09.523ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
234592025-12-16T03:12:09.523ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 201.831787ms iteration = 40 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
234602025-12-16T03:12:09.527ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
234612025-12-16T03:12:09.527ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
234622025-12-16T03:12:09.527ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 205.706926ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
234632025-12-16T03:12:09.541ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
234642025-12-16T03:12:09.541ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 321.398172ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
234652025-12-16T03:12:09.572ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
234662025-12-16T03:12:09.573ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
234672025-12-16T03:12:09.575ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234682025-12-16T03:12:09.578ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
234692025-12-16T03:12:09.578ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
234702025-12-16T03:12:09.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
234712025-12-16T03:12:09.579ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2994091c-bbd4-4473-ae42-b9bc8dd03211 step component = SledAgent step id = Ensure
234722025-12-16T03:12:09.579ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234732025-12-16T03:12:09.580ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234742025-12-16T03:12:09.584ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
234752025-12-16T03:12:09.584ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
234762025-12-16T03:12:09.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
234772025-12-16T03:12:09.585ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
234782025-12-16T03:12:09.585ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234792025-12-16T03:12:09.585ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234802025-12-16T03:12:09.588ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234812025-12-16T03:12:09.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234822025-12-16T03:12:09.590ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234832025-12-16T03:12:09.592ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 412.998942ms iteration = 27 status = {}
234842025-12-16T03:12:09.595ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
234852025-12-16T03:12:09.595ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234862025-12-16T03:12:09.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234872025-12-16T03:12:09.597ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234882025-12-16T03:12:09.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234892025-12-16T03:12:09.598ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234902025-12-16T03:12:09.602ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
234912025-12-16T03:12:09.602ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234922025-12-16T03:12:09.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234932025-12-16T03:12:09.614ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
234942025-12-16T03:12:09.614ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
234952025-12-16T03:12:09.614ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "3399e50c-f396-4e0b-a3d1-f8b150085420", "content-length": "372", "date": "Tue, 16 Dec 2025 03:12:09 GMT"} })
234962025-12-16T03:12:09.614ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
234972025-12-16T03:12:09.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
234982025-12-16T03:12:09.616ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
234992025-12-16T03:12:09.616ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235002025-12-16T03:12:09.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235012025-12-16T03:12:09.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235022025-12-16T03:12:09.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235032025-12-16T03:12:09.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235042025-12-16T03:12:09.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
235052025-12-16T03:12:09.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235062025-12-16T03:12:09.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235072025-12-16T03:12:09.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235082025-12-16T03:12:09.627ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
235092025-12-16T03:12:09.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235102025-12-16T03:12:09.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235112025-12-16T03:12:09.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235122025-12-16T03:12:09.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235132025-12-16T03:12:09.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235142025-12-16T03:12:09.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235152025-12-16T03:12:09.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235162025-12-16T03:12:09.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
235172025-12-16T03:12:09.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235182025-12-16T03:12:09.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235192025-12-16T03:12:09.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235202025-12-16T03:12:09.644ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
235212025-12-16T03:12:09.644ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235222025-12-16T03:12:09.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235232025-12-16T03:12:09.647ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235242025-12-16T03:12:09.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235252025-12-16T03:12:09.648ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
235262025-12-16T03:12:09.648ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235272025-12-16T03:12:09.648ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "2d22896b-7615-4f90-afc7-b2b18d210392", "date": "Tue, 16 Dec 2025 03:12:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
235282025-12-16T03:12:09.648ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
235292025-12-16T03:12:09.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235302025-12-16T03:12:09.649ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1737, tv_nsec: 256088364 } })
235312025-12-16T03:12:09.650ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235322025-12-16T03:12:09.650ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "2ad98fa2-d93b-4106-a888-7bc4517e7b39", "date": "Tue, 16 Dec 2025 03:12:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
235332025-12-16T03:12:09.650ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
235342025-12-16T03:12:09.652ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::9]:5353, [fd00:1122:3344:101::8]:5353] }
235352025-12-16T03:12:09.652ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 473.924315ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
235362025-12-16T03:12:09.653ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
235372025-12-16T03:12:09.667ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
235382025-12-16T03:12:09.667ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
235392025-12-16T03:12:09.667ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "1faf900d-93df-4f12-bb60-8eee4a5b583e", "date": "Tue, 16 Dec 2025 03:12:09 GMT"} })
235402025-12-16T03:12:09.667ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
235412025-12-16T03:12:09.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
235422025-12-16T03:12:09.670ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
235432025-12-16T03:12:09.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
235442025-12-16T03:12:09.674ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
235452025-12-16T03:12:09.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
235462025-12-16T03:12:09.678ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
235472025-12-16T03:12:09.678ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
235482025-12-16T03:12:09.678ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "2e14f0f4-aa42-40e0-b584-ff3a3642ee5f", "date": "Tue, 16 Dec 2025 03:12:09 GMT"} })
235492025-12-16T03:12:09.684ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 362.248479ms iteration = 40 status = {}
235502025-12-16T03:12:09.700ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235512025-12-16T03:12:09.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235522025-12-16T03:12:09.701ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
235532025-12-16T03:12:09.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
235542025-12-16T03:12:09.703ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235552025-12-16T03:12:09.707ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
235562025-12-16T03:12:09.718ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
235572025-12-16T03:12:09.719ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "ef22e0e7-434d-4699-821d-7553472367c4", "date": "Tue, 16 Dec 2025 03:12:09 GMT"} })
235582025-12-16T03:12:09.719ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
235592025-12-16T03:12:09.720ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
235602025-12-16T03:12:09.721ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
235612025-12-16T03:12:09.721ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 483.874491ms iteration = 14 status = {"errors":[],"probes_by_sled":{"b1c76b61-acee-4be6-b582-973c938d6b6e":0}}
235622025-12-16T03:12:09.721ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 41 reason = Dependency
235632025-12-16T03:12:09.721ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
235642025-12-16T03:12:09.721ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
235652025-12-16T03:12:09.722ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "ab62a806-0a4a-4158-809a-75f7577cc2e4", "date": "Tue, 16 Dec 2025 03:12:09 GMT"} }) sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
235662025-12-16T03:12:09.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
235672025-12-16T03:12:09.723ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235682025-12-16T03:12:09.724ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2994091c-bbd4-4473-ae42-b9bc8dd03211 step component = FirewallRules step id = Ensure
235692025-12-16T03:12:09.724ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
235702025-12-16T03:12:09.724ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
235712025-12-16T03:12:09.725ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
235722025-12-16T03:12:09.725ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
235732025-12-16T03:12:09.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
235742025-12-16T03:12:09.726ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
235752025-12-16T03:12:09.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
235762025-12-16T03:12:09.728ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235772025-12-16T03:12:09.729ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
235782025-12-16T03:12:09.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
235792025-12-16T03:12:09.732ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
235802025-12-16T03:12:09.733ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
235812025-12-16T03:12:09.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
235822025-12-16T03:12:09.733ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
235832025-12-16T03:12:09.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
235842025-12-16T03:12:09.734ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
235852025-12-16T03:12:09.734ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "84c7cc6c-ac2e-48d9-87ac-5fafadff1cd3", "content-length": "691", "date": "Tue, 16 Dec 2025 03:12:09 GMT"} })
235862025-12-16T03:12:09.734ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
235872025-12-16T03:12:09.734ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
235882025-12-16T03:12:09.734ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
235892025-12-16T03:12:09.734ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
235902025-12-16T03:12:09.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
235912025-12-16T03:12:09.736ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
235922025-12-16T03:12:09.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
235932025-12-16T03:12:09.737ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 415.090729ms iteration = 27 status = {}
235942025-12-16T03:12:09.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235952025-12-16T03:12:09.743ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
235962025-12-16T03:12:09.743ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
235972025-12-16T03:12:09.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
235982025-12-16T03:12:09.745ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
235992025-12-16T03:12:09.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236002025-12-16T03:12:09.746ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
236012025-12-16T03:12:09.746ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "4e0cd018-5c1a-4a82-a121-003ecea19f6f", "date": "Tue, 16 Dec 2025 03:12:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
236022025-12-16T03:12:09.746ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
236032025-12-16T03:12:09.747ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236042025-12-16T03:12:09.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236052025-12-16T03:12:09.748ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236062025-12-16T03:12:09.749ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "8d72075f-ce9d-4fc4-b8f7-cad663b1d65e", "date": "Tue, 16 Dec 2025 03:12:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
236072025-12-16T03:12:09.749ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
236082025-12-16T03:12:09.751ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "693cfb0b-cff8-4b1c-88df-f3ca641425a0", "date": "Tue, 16 Dec 2025 03:12:09 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
236092025-12-16T03:12:09.751ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
236102025-12-16T03:12:09.752ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1737, tv_nsec: 358782065 } })
236112025-12-16T03:12:09.753ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
236122025-12-16T03:12:09.753ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236132025-12-16T03:12:09.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236142025-12-16T03:12:09.754ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353, [fd00:1122:3344:1::1]:5353] }
236152025-12-16T03:12:09.754ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 575.007725ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
236162025-12-16T03:12:09.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236172025-12-16T03:12:09.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236182025-12-16T03:12:09.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
236192025-12-16T03:12:09.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
236202025-12-16T03:12:09.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236212025-12-16T03:12:09.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236222025-12-16T03:12:09.758ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
236232025-12-16T03:12:09.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
236242025-12-16T03:12:09.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236252025-12-16T03:12:09.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236262025-12-16T03:12:09.760ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236272025-12-16T03:12:09.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236282025-12-16T03:12:09.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236292025-12-16T03:12:09.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236302025-12-16T03:12:09.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236312025-12-16T03:12:09.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
236322025-12-16T03:12:09.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
236332025-12-16T03:12:09.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
236342025-12-16T03:12:09.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236352025-12-16T03:12:09.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236362025-12-16T03:12:09.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
236372025-12-16T03:12:09.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
236382025-12-16T03:12:09.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236392025-12-16T03:12:09.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236402025-12-16T03:12:09.771ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
236412025-12-16T03:12:09.771ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
236422025-12-16T03:12:09.771ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236432025-12-16T03:12:09.771ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
236442025-12-16T03:12:09.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
236452025-12-16T03:12:09.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236462025-12-16T03:12:09.773ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236472025-12-16T03:12:09.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236482025-12-16T03:12:09.773ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
236492025-12-16T03:12:09.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
236502025-12-16T03:12:09.775ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236512025-12-16T03:12:09.776ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236522025-12-16T03:12:09.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236532025-12-16T03:12:09.777ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236542025-12-16T03:12:09.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236552025-12-16T03:12:09.779ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
236562025-12-16T03:12:09.779ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
236572025-12-16T03:12:09.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
236582025-12-16T03:12:09.780ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236592025-12-16T03:12:09.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236602025-12-16T03:12:09.781ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
236612025-12-16T03:12:09.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
236622025-12-16T03:12:09.781ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236632025-12-16T03:12:09.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236642025-12-16T03:12:09.783ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236652025-12-16T03:12:09.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236662025-12-16T03:12:09.784ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236672025-12-16T03:12:09.785ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236682025-12-16T03:12:09.789ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
236692025-12-16T03:12:09.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
236702025-12-16T03:12:09.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
236712025-12-16T03:12:09.790ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
236722025-12-16T03:12:09.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236732025-12-16T03:12:09.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236742025-12-16T03:12:09.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236752025-12-16T03:12:09.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236762025-12-16T03:12:09.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236772025-12-16T03:12:09.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236782025-12-16T03:12:09.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236792025-12-16T03:12:09.795ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
236802025-12-16T03:12:09.795ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
236812025-12-16T03:12:09.798ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e8ba265-db6b-40f8-8355-43bfd0313aaa", "content-length": "372", "date": "Tue, 16 Dec 2025 03:12:09 GMT"} })
236822025-12-16T03:12:09.798ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
236832025-12-16T03:12:09.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
236842025-12-16T03:12:09.801ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
236852025-12-16T03:12:09.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236862025-12-16T03:12:09.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236872025-12-16T03:12:09.806ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236882025-12-16T03:12:09.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236892025-12-16T03:12:09.807ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236902025-12-16T03:12:09.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236912025-12-16T03:12:09.808ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
236922025-12-16T03:12:09.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
236932025-12-16T03:12:09.809ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236942025-12-16T03:12:09.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236952025-12-16T03:12:09.811ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
236962025-12-16T03:12:09.811ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
236972025-12-16T03:12:09.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
236982025-12-16T03:12:09.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236992025-12-16T03:12:09.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
237002025-12-16T03:12:09.814ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
237012025-12-16T03:12:09.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
237022025-12-16T03:12:09.815ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
237032025-12-16T03:12:09.815ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
237042025-12-16T03:12:09.816ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "590c7f7f-2bc6-4336-9368-a732051ffb47", "date": "Tue, 16 Dec 2025 03:12:09 GMT"} })
237052025-12-16T03:12:09.816ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
237062025-12-16T03:12:09.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
237072025-12-16T03:12:09.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
237082025-12-16T03:12:09.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
237092025-12-16T03:12:09.818ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
237102025-12-16T03:12:09.818ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
237112025-12-16T03:12:09.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
237122025-12-16T03:12:09.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
237132025-12-16T03:12:09.821ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
237142025-12-16T03:12:09.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
237152025-12-16T03:12:09.824ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
237162025-12-16T03:12:09.824ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
237172025-12-16T03:12:09.825ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "1e8e7ba7-eba1-48a1-9f79-9a7c1a99149c", "date": "Tue, 16 Dec 2025 03:12:09 GMT"} })
237182025-12-16T03:12:09.827ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 105.262473ms iteration = 41 status = {}
237192025-12-16T03:12:09.828ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
237202025-12-16T03:12:09.828ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
237212025-12-16T03:12:09.829ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
237222025-12-16T03:12:09.829ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
237232025-12-16T03:12:09.829ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
237242025-12-16T03:12:09.829ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
237252025-12-16T03:12:09.829ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
237262025-12-16T03:12:09.829ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
237272025-12-16T03:12:09.829ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
237282025-12-16T03:12:09.829ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
237292025-12-16T03:12:09.830ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
237302025-12-16T03:12:09.830ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
237312025-12-16T03:12:09.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
237322025-12-16T03:12:09.833ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
237332025-12-16T03:12:09.833ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
237342025-12-16T03:12:09.843ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
237352025-12-16T03:12:09.844ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
237362025-12-16T03:12:09.844ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 2427848b-ec34-4c08-9a5b-4a2261abe59d parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
237372025-12-16T03:12:09.844ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
237382025-12-16T03:12:09.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
237392025-12-16T03:12:09.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237402025-12-16T03:12:09.872ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
237412025-12-16T03:12:09.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
237422025-12-16T03:12:09.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
237432025-12-16T03:12:09.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
237442025-12-16T03:12:09.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
237452025-12-16T03:12:09.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
237462025-12-16T03:12:09.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
237472025-12-16T03:12:09.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237482025-12-16T03:12:09.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237492025-12-16T03:12:09.885ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
237502025-12-16T03:12:09.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
237512025-12-16T03:12:09.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
237522025-12-16T03:12:09.887ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
237532025-12-16T03:12:09.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
237542025-12-16T03:12:09.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
237552025-12-16T03:12:09.888ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 2 limit = 5000 usage_percent = 0
237562025-12-16T03:12:09.888ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
237572025-12-16T03:12:09.888ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 667.928953ms iteration = 32 status = {"Unchanged":{"blueprint_count":2,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
237582025-12-16T03:12:09.893ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], Name("nexus"): [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
237592025-12-16T03:12:09.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
237602025-12-16T03:12:09.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
237612025-12-16T03:12:09.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237622025-12-16T03:12:09.897ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
237632025-12-16T03:12:09.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
237642025-12-16T03:12:09.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
237652025-12-16T03:12:09.899ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
237662025-12-16T03:12:09.899ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: 0edf9e41-efd8-4c01-bde0-245ea6b1513d, kind: Service { id: 1a45cd23-3d0a-43eb-adce-62c7b712baf1 }, name: Name("external-dns-1a45cd23-3d0a-43eb-adce-62c7b712baf1"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.6, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 155, 113])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: cb55dc7d-acb9-4e34-824d-0429a5ea94da, kind: Service { id: 8addbd1b-2c59-4c49-96f1-40684614ace0 }, name: Name("external-dns-8addbd1b-2c59-4c49-96f1-40684614ace0"), ip_config: V4(PrivateIpv4Config { ip: 172.30.1.5, subnet: Ipv4Net { addr: 172.30.1.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 243, 93])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 64375bf9-7b07-44a7-9b4a-6f666cffdfdb, kind: Service { id: 1511ffa1-5095-440c-978b-25339cc8be11 }, name: Name("nexus-1511ffa1-5095-440c-978b-25339cc8be11"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.7, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 188])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: a60ccfce-4b8b-4fcf-9d2b-733e95e883bd, kind: Service { id: 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c }, name: Name("nexus-76dbe3e9-13a6-4f57-a61f-1a019ca92d4c"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.5, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 224])), vni: Vni(100), primary: true, slot: 0 }, NetworkInterface { id: d99bea17-19d0-4052-98b0-b6d8305ad59f, kind: Service { id: 8f641c55-8873-4292-9e77-436129b8e195 }, name: Name("nexus-8f641c55-8873-4292-9e77-436129b8e195"), ip_config: V4(PrivateIpv4Config { ip: 172.30.2.6, subnet: Ipv4Net { addr: 172.30.2.0, width: 24 }, transit_ips: [] }), mac: MacAddr(MacAddr6([168, 64, 37, 255, 140, 37])), vni: Vni(100), primary: true, slot: 0 }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
237672025-12-16T03:12:09.899ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
237682025-12-16T03:12:10.043ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), time_created: 2025-12-16T02:59:15.195809Z, time_modified: 2025-12-16T02:59:15.195809Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, is_scrimlet: true, serial_number: "buskin", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::1:0, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
237692025-12-16T03:12:10.043ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
237702025-12-16T03:12:10.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
237712025-12-16T03:12:10.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
237722025-12-16T03:12:10.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
237732025-12-16T03:12:10.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
237742025-12-16T03:12:10.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237752025-12-16T03:12:10.081ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
237762025-12-16T03:12:10.081ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
237772025-12-16T03:12:10.083ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "27bd4abf-1381-4c76-b1d6-d8dd1ea8f361", "date": "Tue, 16 Dec 2025 03:12:10 GMT"} })
237782025-12-16T03:12:10.085ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:506
237792025-12-16T03:12:10.085ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2994091c-bbd4-4473-ae42-b9bc8dd03211 step component = Dns step id = Ensure
237802025-12-16T03:12:10.085ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
237812025-12-16T03:12:10.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
237822025-12-16T03:12:10.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
237832025-12-16T03:12:10.086ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
237842025-12-16T03:12:10.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
237852025-12-16T03:12:10.088ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237862025-12-16T03:12:10.091ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
237872025-12-16T03:12:10.129ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
237882025-12-16T03:12:10.129ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
237892025-12-16T03:12:10.129ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
237902025-12-16T03:12:10.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
237912025-12-16T03:12:10.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
237922025-12-16T03:12:10.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237932025-12-16T03:12:10.134ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
237942025-12-16T03:12:10.134ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
237952025-12-16T03:12:10.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
237962025-12-16T03:12:10.137ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
237972025-12-16T03:12:10.137ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
237982025-12-16T03:12:10.137ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
237992025-12-16T03:12:10.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
238002025-12-16T03:12:10.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238012025-12-16T03:12:10.142ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "6fff6c8f-6a36-456f-a4f9-abbc4dc1ebbe", "date": "Tue, 16 Dec 2025 03:12:10 GMT"} })
238022025-12-16T03:12:10.142ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
238032025-12-16T03:12:10.142ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
238042025-12-16T03:12:10.142ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
238052025-12-16T03:12:10.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
238062025-12-16T03:12:10.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
238072025-12-16T03:12:10.142ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fe6408a-859d-441f-979f-751aaf4c35ba", "content-length": "682", "date": "Tue, 16 Dec 2025 03:12:10 GMT"} })
238082025-12-16T03:12:10.142ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
238092025-12-16T03:12:10.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
238102025-12-16T03:12:10.144ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = b8fba208-d31b-4aee-a17d-2b9869fcd40c dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
238112025-12-16T03:12:10.144ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
238122025-12-16T03:12:10.144ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
238132025-12-16T03:12:10.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
238142025-12-16T03:12:10.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
238152025-12-16T03:12:10.145ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
238162025-12-16T03:12:10.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
238172025-12-16T03:12:10.146ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
238182025-12-16T03:12:10.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
238192025-12-16T03:12:10.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238202025-12-16T03:12:10.150ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
238212025-12-16T03:12:10.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
238222025-12-16T03:12:10.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
238232025-12-16T03:12:10.152ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
238242025-12-16T03:12:10.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
238252025-12-16T03:12:10.158ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 838.654996ms iteration = 27 status = {}
238262025-12-16T03:12:10.195ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
238272025-12-16T03:12:10.195ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
238282025-12-16T03:12:10.195ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
238292025-12-16T03:12:10.195ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
238302025-12-16T03:12:10.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
238312025-12-16T03:12:10.197ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238322025-12-16T03:12:10.201ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
238332025-12-16T03:12:10.201ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
238342025-12-16T03:12:10.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
238352025-12-16T03:12:10.205ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
238362025-12-16T03:12:10.205ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 0 version = 2
238372025-12-16T03:12:10.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
238382025-12-16T03:12:10.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
238392025-12-16T03:12:10.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238402025-12-16T03:12:10.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
238412025-12-16T03:12:10.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
238422025-12-16T03:12:10.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
238432025-12-16T03:12:10.212ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 17d42677-d554-4b75-a164-ff181564d0aa dns_zone_name = oxide.test found_so_far = 4 version = 2
238442025-12-16T03:12:10.212ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
238452025-12-16T03:12:10.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
238462025-12-16T03:12:10.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
238472025-12-16T03:12:10.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238482025-12-16T03:12:10.217ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
238492025-12-16T03:12:10.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
238502025-12-16T03:12:10.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
238512025-12-16T03:12:10.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
238522025-12-16T03:12:10.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
238532025-12-16T03:12:10.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
238542025-12-16T03:12:10.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
238552025-12-16T03:12:10.221ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238562025-12-16T03:12:10.224ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
238572025-12-16T03:12:10.225ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
238582025-12-16T03:12:10.225ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
238592025-12-16T03:12:10.225ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2994091c-bbd4-4473-ae42-b9bc8dd03211 step component = OmicronZones step id = Cleanup
238602025-12-16T03:12:10.225ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2994091c-bbd4-4473-ae42-b9bc8dd03211 step component = OmicronZones step id = Cleanup
238612025-12-16T03:12:10.225ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2994091c-bbd4-4473-ae42-b9bc8dd03211 step component = PhysicalDisks step id = Cleanup
238622025-12-16T03:12:10.226ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2994091c-bbd4-4473-ae42-b9bc8dd03211 step component = Clickhouse step id = Ensure
238632025-12-16T03:12:10.226ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2994091c-bbd4-4473-ae42-b9bc8dd03211 step component = Clickhouse step id = Ensure
238642025-12-16T03:12:10.256ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
238652025-12-16T03:12:10.260ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "a07f482d-6139-4d4b-bbcc-25e09d52f214", "date": "Tue, 16 Dec 2025 03:12:10 GMT"} })
238662025-12-16T03:12:10.261ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2994091c-bbd4-4473-ae42-b9bc8dd03211 step component = SupportBundles step id = Cleanup
238672025-12-16T03:12:10.261ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
238682025-12-16T03:12:10.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
238692025-12-16T03:12:10.263ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238702025-12-16T03:12:10.264ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
238712025-12-16T03:12:10.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
238722025-12-16T03:12:10.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
238732025-12-16T03:12:10.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
238742025-12-16T03:12:10.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
238752025-12-16T03:12:10.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238762025-12-16T03:12:10.270ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
238772025-12-16T03:12:10.310ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2994091c-bbd4-4473-ae42-b9bc8dd03211 step component = OmicronZones step id = Cleanup
238782025-12-16T03:12:10.310ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
238792025-12-16T03:12:10.310ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
238802025-12-16T03:12:10.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
238812025-12-16T03:12:10.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
238822025-12-16T03:12:10.311ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238832025-12-16T03:12:10.313ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
238842025-12-16T03:12:10.313ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
238852025-12-16T03:12:10.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
238862025-12-16T03:12:10.315ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
238872025-12-16T03:12:10.315ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(a91734db-9ac7-4793-9c32-ddbb21f8d986 (blueprint), false)
238882025-12-16T03:12:10.315ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2994091c-bbd4-4473-ae42-b9bc8dd03211 step component = Cockroach step id = Ensure
238892025-12-16T03:12:10.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
238902025-12-16T03:12:10.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
238912025-12-16T03:12:10.362ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
238922025-12-16T03:12:10.362ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 2994091c-bbd4-4473-ae42-b9bc8dd03211 step component = MgsUpdates step id = Ensure
238932025-12-16T03:12:10.362ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 41 reason = Dependency
238942025-12-16T03:12:10.362ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
238952025-12-16T03:12:10.362ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 16 reason = Dependency
238962025-12-16T03:12:10.362ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
238972025-12-16T03:12:10.363ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.183763288s iteration = 15 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"739f8080-59a8-48dc-a159-089fbbf06538","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"739f8080-59a8-48dc-a159-089fbbf06538","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":20681,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":98154603,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":98154563,"secs":0}},"event_index":1,"execution_id":"739f8080-59a8-48dc-a159-089fbbf06538","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":98193604,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":223810525,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":223810485,"secs":0}},"event_index":2,"execution_id":"739f8080-59a8-48dc-a159-089fbbf06538","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":322019210,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":78490475,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":78490425,"secs":0}},"event_index":3,"execution_id":"739f8080-59a8-48dc-a159-089fbbf06538","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":400532545,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":144537151,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":144537121,"secs":0}},"event_index":4,"execution_id":"739f8080-59a8-48dc-a159-089fbbf06538","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":545093626,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":361012484,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":361012444,"secs":0}},"event_index":5,"execution_id":"739f8080-59a8-48dc-a159-089fbbf06538","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":906118761,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":140607041,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":140607001,"secs":0}},"event_index":6,"execution_id":"739f8080-59a8-48dc-a159-089fbbf06538","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":46741673,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3770,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":3730,"secs":0}},"event_index":7,"execution_id":"739f8080-59a8-48dc-a159-089fbbf06538","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":46759263,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2270,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2220,"secs":0}},"event_index":8,"execution_id":"739f8080-59a8-48dc-a159-089fbbf06538","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":46763923,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3320,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3280,"secs":0}},"event_index":9,"execution_id":"739f8080-59a8-48dc-a159-089fbbf06538","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":46769803,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2170,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":2120,"secs":0}},"event_index":10,"execution_id":"739f8080-59a8-48dc-a159-089fbbf06538","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":46774533,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":35842434,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":35842404,"secs":0}},"event_index":11,"execution_id":"739f8080-59a8-48dc-a159-089fbbf06538","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":82618607,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":48591086,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":48591046,"secs":0}},"event_index":12,"execution_id":"739f8080-59a8-48dc-a159-089fbbf06538","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":131220763,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4745550,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4745510,"secs":0}},"event_index":13,"execution_id":"739f8080-59a8-48dc-a159-089fbbf06538","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":135976733,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":47448922,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":47448892,"secs":0}},"event_index":14,"execution_id":"739f8080-59a8-48dc-a159-089fbbf06538","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":183436905,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5930,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5890,"secs":0}},"event_index":15,"execution_id":"739f8080-59a8-48dc-a159-089fbbf06538","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":183451225,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986"}
238982025-12-16T03:12:10.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
238992025-12-16T03:12:10.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
239002025-12-16T03:12:10.364ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
239012025-12-16T03:12:10.364ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
239022025-12-16T03:12:10.367ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
239032025-12-16T03:12:10.367ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
239042025-12-16T03:12:10.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
239052025-12-16T03:12:10.367ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
239062025-12-16T03:12:10.367ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
239072025-12-16T03:12:10.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
239082025-12-16T03:12:10.403ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
239092025-12-16T03:12:10.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
239102025-12-16T03:12:10.405ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
239112025-12-16T03:12:10.407ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
239122025-12-16T03:12:10.407ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
239132025-12-16T03:12:10.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
239142025-12-16T03:12:10.409ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
239152025-12-16T03:12:10.409ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 46.701871ms iteration = 41 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
239162025-12-16T03:12:10.935ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(9731e6a4-dd37-464b-8145-974fc4f4be58, 0), (4cf7dcb3-4693-4eaf-af85-9f40dc98c412, 0), (ed689c21-be41-443a-8f12-337f3395c924, 0), (ad48459f-45e8-4b99-938a-cf5f22361b26, 0)] collection_id = 9731e6a4-dd37-464b-8145-974fc4f4be58
239172025-12-16T03:12:10.936ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
239182025-12-16T03:12:10.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
239192025-12-16T03:12:11.071ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 278 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 51e9e425-a19f-4864-ab17-0d0bae581184 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
239202025-12-16T03:12:11.911ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 9731e6a4-dd37-464b-8145-974fc4f4be58 file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
239212025-12-16T03:12:11.911ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
239222025-12-16T03:12:11.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
239232025-12-16T03:12:12.657ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(ed689c21-be41-443a-8f12-337f3395c924, 0), (ad48459f-45e8-4b99-938a-cf5f22361b26, 0), (60390ee0-9ebf-417a-b74d-f8a1c4e08914, 0)]
239242025-12-16T03:12:12.658ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1740, tv_nsec: 264118336 } })
239252025-12-16T03:12:12.688ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("20488683-5778-4090-b36e-e6f0d80a682b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("35dbaad1-8536-44a0-8a0a-35d7c39dcfbb.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("b57fa226-2428-4a54-b85d-60409e28b484.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("d812a3e2-48ae-4812-a1cc-7eb7623e4e00.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("e19ae8d1-5396-4385-8758-ea60b907a565.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::5)) }], valid_until: Instant { tv_sec: 1740, tv_nsec: 294919936 } })
239262025-12-16T03:12:12.689ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
239272025-12-16T03:12:12.689ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
239282025-12-16T03:12:12.689ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
239292025-12-16T03:12:12.690ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "f334d692-0cc2-4db5-ac9a-6590dc7b8a52", "content-length": "476", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239302025-12-16T03:12:12.690ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
239312025-12-16T03:12:12.690ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "33f9cc06-2a78-4815-b31b-5d571730bdd5", "content-length": "734", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239322025-12-16T03:12:12.690ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
239332025-12-16T03:12:12.690ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1594bba8-3499-43b3-ba09-e9abba889886", "content-length": "111", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239342025-12-16T03:12:12.690ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
239352025-12-16T03:12:12.691ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "664ada44-db66-42c4-aa78-e3a5fca6549c", "content-length": "111", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239362025-12-16T03:12:12.691ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
239372025-12-16T03:12:12.691ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "13ef3553-bb85-4b91-ae20-49314f18f330", "content-length": "163", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239382025-12-16T03:12:12.691ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
239392025-12-16T03:12:12.692ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0392116-e72f-46b7-a901-578e75fe1a2d", "content-length": "163", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239402025-12-16T03:12:12.692ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
239412025-12-16T03:12:12.692ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2df72cba-79b3-4bef-bda7-e5512154e6c8", "content-length": "166", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239422025-12-16T03:12:12.692ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
239432025-12-16T03:12:12.693ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3180bd37-aac0-4a09-8177-e007d034bfc0", "content-length": "166", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239442025-12-16T03:12:12.693ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
239452025-12-16T03:12:12.693ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6fc5cf8e-1b1d-48f1-8388-0371de96d8cd", "content-length": "702", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239462025-12-16T03:12:12.693ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
239472025-12-16T03:12:12.693ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfffaf99-de2a-4ad4-98cc-f7eb151a7b1c", "content-length": "727", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239482025-12-16T03:12:12.693ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
239492025-12-16T03:12:12.693ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "52f677ce-80e5-45d1-afbb-8f3c7763d9e6", "content-length": "729", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239502025-12-16T03:12:12.693ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
239512025-12-16T03:12:12.693ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce6c9b48-0d70-4693-b0f1-3fa5195cd7a7", "content-length": "728", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239522025-12-16T03:12:12.693ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
239532025-12-16T03:12:12.694ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e620f41-5e51-44e5-8fe2-332c9d1ccd4e", "content-length": "722", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239542025-12-16T03:12:12.694ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
239552025-12-16T03:12:12.694ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "aef50558-5a53-4b6e-8502-00640e86f2c8", "content-length": "10", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239562025-12-16T03:12:12.694ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
239572025-12-16T03:12:12.694ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b5ddb40-7807-456f-80a1-46b168766cd4", "content-length": "144", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239582025-12-16T03:12:12.694ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
239592025-12-16T03:12:12.694ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "07fc1526-ee5c-4f92-bb64-a3afdc76bc00", "content-length": "147", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239602025-12-16T03:12:12.694ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
239612025-12-16T03:12:12.695ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e76988fb-e16b-42b1-aa91-92d3440834e8", "content-length": "109", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239622025-12-16T03:12:12.695ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
239632025-12-16T03:12:12.695ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cc47158-f69c-4da2-ac50-f1913058fab6", "content-length": "109", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239642025-12-16T03:12:12.695ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
239652025-12-16T03:12:12.696ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "37670d10-b87e-49ea-a49f-6514b4fc1f7b", "content-length": "163", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239662025-12-16T03:12:12.696ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
239672025-12-16T03:12:12.696ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5edbc3d7-cb4f-4c45-a73a-b773966722bb", "content-length": "163", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239682025-12-16T03:12:12.696ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
239692025-12-16T03:12:12.696ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c771b13-106d-4083-8ce9-c04065a74533", "content-length": "166", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239702025-12-16T03:12:12.696ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
239712025-12-16T03:12:12.697ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c47e16ee-22e8-48ec-b956-f9c550c55948", "content-length": "166", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239722025-12-16T03:12:12.697ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
239732025-12-16T03:12:12.697ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7d5ca17-039d-418a-a317-4d85aaf7a29f", "content-length": "702", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239742025-12-16T03:12:12.697ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
239752025-12-16T03:12:12.697ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7f85bd5-4943-454d-ab25-a11e72dd16f6", "content-length": "727", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239762025-12-16T03:12:12.697ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
239772025-12-16T03:12:12.698ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3622c5d-3f3f-432f-a422-464480aa1f1d", "content-length": "729", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239782025-12-16T03:12:12.698ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
239792025-12-16T03:12:12.698ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "420292f9-142b-45e5-8186-d890dd62bda0", "content-length": "728", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239802025-12-16T03:12:12.698ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
239812025-12-16T03:12:12.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
239822025-12-16T03:12:12.700ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
239832025-12-16T03:12:12.702ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
239842025-12-16T03:12:12.702ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
239852025-12-16T03:12:12.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
239862025-12-16T03:12:12.704ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
239872025-12-16T03:12:12.707ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
239882025-12-16T03:12:12.707ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
239892025-12-16T03:12:12.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
239902025-12-16T03:12:12.744ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
239912025-12-16T03:12:12.744ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
239922025-12-16T03:12:12.854ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "cef3df35-40f0-438b-a801-5b66e190c576", "content-length": "95845", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
239932025-12-16T03:12:12.856ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
239942025-12-16T03:12:12.856ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
239952025-12-16T03:12:12.856ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
239962025-12-16T03:12:12.856ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
239972025-12-16T03:12:12.856ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
239982025-12-16T03:12:12.856ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
239992025-12-16T03:12:12.856ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
240002025-12-16T03:12:12.856ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
240012025-12-16T03:12:12.856ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "750c8eab-353e-42fe-b77d-5a5680e96f3e", "content-length": "64", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
240022025-12-16T03:12:12.856ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
240032025-12-16T03:12:12.856ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4903affe-df23-4d19-858c-c287b99c3509", "content-length": "64", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
240042025-12-16T03:12:12.856ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
240052025-12-16T03:12:12.856ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5ce5727-6f02-4c36-a597-820d9a4db76a", "content-length": "64", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
240062025-12-16T03:12:12.856ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "afced5c6-d20f-4698-9db5-c2269adaad15", "content-length": "64", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
240072025-12-16T03:12:12.856ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a838558c-9831-4788-9a34-095cffedda61", "content-length": "64", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
240082025-12-16T03:12:12.856ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
240092025-12-16T03:12:12.856ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
240102025-12-16T03:12:12.856ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
240112025-12-16T03:12:12.894ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7c4236d-b487-4cc2-8f17-1f044290fd15", "content-length": "442127", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
240122025-12-16T03:12:12.894ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "6abeb1f2-c835-4ef0-a8a1-736ea82d6016", "content-length": "412395", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
240132025-12-16T03:12:12.895ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "1bdd62de-bdcc-40f2-b4a0-8686351f93fd", "content-length": "414573", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
240142025-12-16T03:12:12.896ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "47a42107-7dae-4057-bd4d-22b3aff4427b", "content-length": "413743", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
240152025-12-16T03:12:12.897ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "39cc8139-89cb-4f3c-bbd6-cfdf0ff05464", "content-length": "414434", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
240162025-12-16T03:12:12.900ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
240172025-12-16T03:12:12.900ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
240182025-12-16T03:12:12.900ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
240192025-12-16T03:12:12.900ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
240202025-12-16T03:12:12.900ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
240212025-12-16T03:12:12.932ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 5f72890f-30aa-4ce1-8d39-5b7bd438f00c (omicron_zone)
240222025-12-16T03:12:12.932ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
240232025-12-16T03:12:12.935ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "1287ee77-f645-47e1-9a5f-69a276a59311", "content-length": "124", "date": "Tue, 16 Dec 2025 03:12:12 GMT"} })
240242025-12-16T03:12:12.937ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
240252025-12-16T03:12:13.028ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
240262025-12-16T03:12:13.028ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
240272025-12-16T03:12:13.029ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c3a38ab-59d5-439a-8f7b-0e47cd703ebb", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:12:13 GMT"} })
240282025-12-16T03:12:13.029ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 49a51a59-9919-48ae-8182-4070a8430160 (omicron_zone)
240292025-12-16T03:12:13.031ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
240302025-12-16T03:12:13.031ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
240312025-12-16T03:12:13.031ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "307119a3-f7e8-458c-a1d4-10b962ca50ae", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:12:13 GMT"} })
240322025-12-16T03:12:13.031ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 6af60386-8771-4401-9d4d-e93774e519f0 (omicron_zone)
240332025-12-16T03:12:13.033ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
240342025-12-16T03:12:13.033ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
240352025-12-16T03:12:13.033ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae9f9abe-10d8-4a7a-9a33-ef07f96ab617", "content-length": "9552", "date": "Tue, 16 Dec 2025 03:12:13 GMT"} })
240362025-12-16T03:12:13.033ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 87dd4f81-0d39-4f7a-962b-d01d4c876a84 (omicron_zone)
240372025-12-16T03:12:13.035ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
240382025-12-16T03:12:13.035ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
240392025-12-16T03:12:13.037ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
240402025-12-16T03:12:13.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
240412025-12-16T03:12:13.039ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240422025-12-16T03:12:13.042ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
240432025-12-16T03:12:13.042ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
240442025-12-16T03:12:13.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
240452025-12-16T03:12:13.517ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = f36d6c1a-1a48-4630-a7cf-001515ed9bca file = nexus/db-queries/src/db/datastore/inventory.rs:1613
240462025-12-16T03:12:13.517ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = f36d6c1a-1a48-4630-a7cf-001515ed9bca time_started = 2025-12-16 03:12:12.689560 UTC
240472025-12-16T03:12:13.517ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 69 reason = Dependency
240482025-12-16T03:12:13.517ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 46 reason = Dependency
240492025-12-16T03:12:13.517ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
240502025-12-16T03:12:13.517ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
240512025-12-16T03:12:13.517ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 21.331\u{b5}s iteration = 46 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
240522025-12-16T03:12:13.517ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 3.155140115s iteration = 16 status = {"collection_id":"f36d6c1a-1a48-4630-a7cf-001515ed9bca","time_done":"2025-12-16 03:12:13.035778 UTC","time_started":"2025-12-16 03:12:12.689560 UTC"}
240532025-12-16T03:12:13.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
240542025-12-16T03:12:13.520ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240552025-12-16T03:12:13.523ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
240562025-12-16T03:12:13.523ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
240572025-12-16T03:12:13.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
240582025-12-16T03:12:13.711ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
240592025-12-16T03:12:13.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
240602025-12-16T03:12:13.897ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 20 reason = Dependency
240612025-12-16T03:12:13.897ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 379.499429ms iteration = 69 status = {"Loaded":{"collection_id":"76002c35-1c9e-4f34-86ef-e4a5ebd28492","time_loaded":"2025-12-16T03:12:13.517827338Z","time_started":"2025-12-16T03:12:12.690353Z"}}
240622025-12-16T03:12:13.897ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 44 reason = Dependency
240632025-12-16T03:12:13.897ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 33 reason = Dependency
240642025-12-16T03:12:13.897ZDEBGnexus (BackgroundTasks): activating background_task = multicast_reconciler iteration = 47 reason = Dependency
240652025-12-16T03:12:13.897ZINFOnexus (BackgroundTasks): multicast group reconciler not enabled background_task = multicast_reconciler file = nexus/src/app/background/tasks/multicast/mod.rs:354
240662025-12-16T03:12:13.897ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
240672025-12-16T03:12:13.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
240682025-12-16T03:12:13.897ZDEBGnexus (BackgroundTasks): activation complete background_task = multicast_reconciler elapsed = 15.46\u{b5}s iteration = 47 status = {"disabled":true,"errors":[],"groups_created":0,"groups_deleted":0,"groups_verified":0,"members_deleted":0,"members_processed":0}
240692025-12-16T03:12:13.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
240702025-12-16T03:12:13.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
240712025-12-16T03:12:13.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
240722025-12-16T03:12:13.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
240732025-12-16T03:12:13.899ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240742025-12-16T03:12:13.899ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240752025-12-16T03:12:13.899ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240762025-12-16T03:12:13.902ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
240772025-12-16T03:12:13.902ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
240782025-12-16T03:12:13.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
240792025-12-16T03:12:13.903ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
240802025-12-16T03:12:13.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
240812025-12-16T03:12:13.903ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
240822025-12-16T03:12:13.903ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
240832025-12-16T03:12:13.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
240842025-12-16T03:12:13.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
240852025-12-16T03:12:13.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240862025-12-16T03:12:13.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240872025-12-16T03:12:13.905ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
240882025-12-16T03:12:13.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
240892025-12-16T03:12:13.906ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
240902025-12-16T03:12:13.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
240912025-12-16T03:12:13.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
240922025-12-16T03:12:13.907ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240932025-12-16T03:12:13.907ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
240942025-12-16T03:12:13.907ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
240952025-12-16T03:12:13.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
240962025-12-16T03:12:13.908ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
240972025-12-16T03:12:13.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
240982025-12-16T03:12:13.909ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
240992025-12-16T03:12:13.909ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
241002025-12-16T03:12:13.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
241012025-12-16T03:12:13.910ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241022025-12-16T03:12:13.912ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
241032025-12-16T03:12:13.912ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
241042025-12-16T03:12:13.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
241052025-12-16T03:12:13.914ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241062025-12-16T03:12:13.917ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
241072025-12-16T03:12:13.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
241082025-12-16T03:12:13.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
241092025-12-16T03:12:13.924ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
241102025-12-16T03:12:13.924ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
241112025-12-16T03:12:13.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
241122025-12-16T03:12:13.926ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241132025-12-16T03:12:13.930ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
241142025-12-16T03:12:13.930ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
241152025-12-16T03:12:13.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
241162025-12-16T03:12:13.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241172025-12-16T03:12:13.935ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
241182025-12-16T03:12:13.935ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
241192025-12-16T03:12:13.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
241202025-12-16T03:12:13.939ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
241212025-12-16T03:12:13.939ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 42.121883ms iteration = 20 status = {"blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","inventory_collection_id":"76002c35-1c9e-4f34-86ef-e4a5ebd28492","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
241222025-12-16T03:12:13.971ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 73.820415ms iteration = 44 status = {"physical_disks_added":0}
241232025-12-16T03:12:14.028ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241242025-12-16T03:12:14.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241252025-12-16T03:12:14.029ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241262025-12-16T03:12:14.032ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
241272025-12-16T03:12:14.032ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241282025-12-16T03:12:14.032ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241292025-12-16T03:12:14.033ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241302025-12-16T03:12:14.035ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
241312025-12-16T03:12:14.035ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241322025-12-16T03:12:14.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241332025-12-16T03:12:14.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241342025-12-16T03:12:14.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241352025-12-16T03:12:14.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241362025-12-16T03:12:14.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241372025-12-16T03:12:14.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241382025-12-16T03:12:14.084ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
241392025-12-16T03:12:14.084ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241402025-12-16T03:12:14.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241412025-12-16T03:12:14.087ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241422025-12-16T03:12:14.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241432025-12-16T03:12:14.088ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241442025-12-16T03:12:14.091ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
241452025-12-16T03:12:14.091ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241462025-12-16T03:12:14.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241472025-12-16T03:12:14.092ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241482025-12-16T03:12:14.095ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
241492025-12-16T03:12:14.095ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241502025-12-16T03:12:14.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241512025-12-16T03:12:14.098ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241522025-12-16T03:12:14.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241532025-12-16T03:12:14.099ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241542025-12-16T03:12:14.102ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: ef0a4eab-b253-47b6-9a59-ed7fb5abda4e, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
241552025-12-16T03:12:14.102ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241562025-12-16T03:12:14.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241572025-12-16T03:12:14.104ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241582025-12-16T03:12:14.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241592025-12-16T03:12:14.105ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241602025-12-16T03:12:14.108ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
241612025-12-16T03:12:14.108ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241622025-12-16T03:12:14.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241632025-12-16T03:12:14.110ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241642025-12-16T03:12:14.111ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241652025-12-16T03:12:14.112ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241662025-12-16T03:12:14.115ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
241672025-12-16T03:12:14.115ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241682025-12-16T03:12:14.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241692025-12-16T03:12:14.136ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241702025-12-16T03:12:14.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241712025-12-16T03:12:14.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241722025-12-16T03:12:14.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241732025-12-16T03:12:14.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241742025-12-16T03:12:14.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
241752025-12-16T03:12:14.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241762025-12-16T03:12:14.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241772025-12-16T03:12:14.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241782025-12-16T03:12:14.149ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
241792025-12-16T03:12:14.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241802025-12-16T03:12:14.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241812025-12-16T03:12:14.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241822025-12-16T03:12:14.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241832025-12-16T03:12:14.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241842025-12-16T03:12:14.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241852025-12-16T03:12:14.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241862025-12-16T03:12:14.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
241872025-12-16T03:12:14.159ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241882025-12-16T03:12:14.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241892025-12-16T03:12:14.160ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241902025-12-16T03:12:14.164ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: bc94102e-b6a4-4d3b-850b-bf569b518d27, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
241912025-12-16T03:12:14.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241922025-12-16T03:12:14.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241932025-12-16T03:12:14.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241942025-12-16T03:12:14.166ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241952025-12-16T03:12:14.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
241962025-12-16T03:12:14.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
241972025-12-16T03:12:14.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241982025-12-16T03:12:14.173ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
241992025-12-16T03:12:14.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
242002025-12-16T03:12:14.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
242012025-12-16T03:12:14.175ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
242022025-12-16T03:12:14.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
242032025-12-16T03:12:14.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242042025-12-16T03:12:14.179ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
242052025-12-16T03:12:14.224ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
242062025-12-16T03:12:14.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
242072025-12-16T03:12:14.227ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
242082025-12-16T03:12:14.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
242092025-12-16T03:12:14.229ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242102025-12-16T03:12:14.232ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
242112025-12-16T03:12:14.232ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
242122025-12-16T03:12:14.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
242132025-12-16T03:12:14.234ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
242142025-12-16T03:12:14.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
242152025-12-16T03:12:14.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242162025-12-16T03:12:14.238ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
242172025-12-16T03:12:14.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
242182025-12-16T03:12:14.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
242192025-12-16T03:12:14.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
242202025-12-16T03:12:14.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
242212025-12-16T03:12:14.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242222025-12-16T03:12:14.245ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
242232025-12-16T03:12:14.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
242242025-12-16T03:12:14.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
242252025-12-16T03:12:14.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
242262025-12-16T03:12:14.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
242272025-12-16T03:12:14.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242282025-12-16T03:12:14.250ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
242292025-12-16T03:12:14.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
242302025-12-16T03:12:14.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
242312025-12-16T03:12:14.289ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
242322025-12-16T03:12:14.289ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
242332025-12-16T03:12:14.289ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 8ab26d08-7ac2-48c4-a93c-d290b76c457f parent_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 sled_id = b1c76b61-acee-4be6-b582-973c938d6b6e
242342025-12-16T03:12:14.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
242352025-12-16T03:12:14.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
242362025-12-16T03:12:14.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242372025-12-16T03:12:14.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
242382025-12-16T03:12:14.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
242392025-12-16T03:12:14.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
242402025-12-16T03:12:14.297ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 2 limit = 5000 usage_percent = 0
242412025-12-16T03:12:14.297ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:257 parent_blueprint_id = a91734db-9ac7-4793-9c32-ddbb21f8d986
242422025-12-16T03:12:14.297ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 400.050679ms iteration = 33 status = {"Unchanged":{"blueprint_count":2,"limit":5000,"parent_blueprint_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled b1c76b61-acee-4be6-b582-973c938d6b6e: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
242432025-12-16T03:12:14.320ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 158 reason = Timeout
242442025-12-16T03:12:14.320ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
242452025-12-16T03:12:14.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
242462025-12-16T03:12:14.322ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242472025-12-16T03:12:14.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
242482025-12-16T03:12:14.325ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
242492025-12-16T03:12:14.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
242502025-12-16T03:12:14.328ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
242512025-12-16T03:12:14.328ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.325353ms iteration = 158 status = {"config_updated":false}
242522025-12-16T03:12:19.321ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 80 reason = Timeout
242532025-12-16T03:12:19.321ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 80 reason = Timeout
242542025-12-16T03:12:19.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
242552025-12-16T03:12:19.321ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
242562025-12-16T03:12:19.321ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 159 reason = Timeout
242572025-12-16T03:12:19.321ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
242582025-12-16T03:12:19.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
242592025-12-16T03:12:19.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
242602025-12-16T03:12:19.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
242612025-12-16T03:12:19.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242622025-12-16T03:12:19.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242632025-12-16T03:12:19.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
242642025-12-16T03:12:19.327ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
242652025-12-16T03:12:19.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
242662025-12-16T03:12:19.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
242672025-12-16T03:12:19.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
242682025-12-16T03:12:19.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
242692025-12-16T03:12:19.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
242702025-12-16T03:12:19.330ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
242712025-12-16T03:12:19.330ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.647169ms iteration = 159 status = {"config_updated":false}
242722025-12-16T03:12:19.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
242732025-12-16T03:12:19.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242742025-12-16T03:12:19.334ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
242752025-12-16T03:12:19.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
242762025-12-16T03:12:19.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
242772025-12-16T03:12:19.352ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
242782025-12-16T03:12:19.352ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 31.147825ms iteration = 80 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
242792025-12-16T03:12:19.361ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
242802025-12-16T03:12:19.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
242812025-12-16T03:12:19.363ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 41.854371ms iteration = 80 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
242822025-12-16T03:12:21.071ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 277 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = 7194f962-fcb1-42bd-8469-e4cd152ec910 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
242832025-12-16T03:12:24.320ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 54 reason = Timeout
242842025-12-16T03:12:24.320ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 70 reason = Timeout
242852025-12-16T03:12:24.320ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
242862025-12-16T03:12:24.320ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 54 reason = Timeout
242872025-12-16T03:12:24.320ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
242882025-12-16T03:12:24.320ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
242892025-12-16T03:12:24.320ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 160 reason = Timeout
242902025-12-16T03:12:24.320ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
242912025-12-16T03:12:24.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
242922025-12-16T03:12:24.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
242932025-12-16T03:12:24.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
242942025-12-16T03:12:24.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
242952025-12-16T03:12:24.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
242962025-12-16T03:12:24.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
242972025-12-16T03:12:24.322ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242982025-12-16T03:12:24.322ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242992025-12-16T03:12:24.322ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
243002025-12-16T03:12:24.323ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
243012025-12-16T03:12:24.326ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
243022025-12-16T03:12:24.326ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
243032025-12-16T03:12:24.326ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
243042025-12-16T03:12:24.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
243052025-12-16T03:12:24.327ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 7.231822ms iteration = 54 status = "NoSitrep"
243062025-12-16T03:12:24.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
243072025-12-16T03:12:24.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
243082025-12-16T03:12:24.327ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
243092025-12-16T03:12:24.327ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
243102025-12-16T03:12:24.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
243112025-12-16T03:12:24.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
243122025-12-16T03:12:24.329ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
243132025-12-16T03:12:24.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
243142025-12-16T03:12:24.329ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
243152025-12-16T03:12:24.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
243162025-12-16T03:12:24.330ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 76002c35-1c9e-4f34-86ef-e4a5ebd28492 original_time_started = 2025-12-16 03:12:12.690353 UTC
243172025-12-16T03:12:24.330ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 9.43076ms iteration = 70 status = {"Loaded":{"collection_id":"76002c35-1c9e-4f34-86ef-e4a5ebd28492","time_loaded":"2025-12-16T03:12:24.320632688Z","time_started":"2025-12-16T03:12:12.690353Z"}}
243182025-12-16T03:12:24.330ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
243192025-12-16T03:12:24.330ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 9.515251ms iteration = 160 status = {"config_updated":false}
243202025-12-16T03:12:24.331ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
243212025-12-16T03:12:24.332ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
243222025-12-16T03:12:24.333ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
243232025-12-16T03:12:24.333ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 12.934094ms iteration = 54 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
243242025-12-16T03:12:29.321ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 81 reason = Timeout
243252025-12-16T03:12:29.321ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 81 reason = Timeout
243262025-12-16T03:12:29.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
243272025-12-16T03:12:29.321ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
243282025-12-16T03:12:29.321ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 161 reason = Timeout
243292025-12-16T03:12:29.321ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
243302025-12-16T03:12:29.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
243312025-12-16T03:12:29.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
243322025-12-16T03:12:29.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
243332025-12-16T03:12:29.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
243342025-12-16T03:12:29.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
243352025-12-16T03:12:29.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
243362025-12-16T03:12:29.327ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
243372025-12-16T03:12:29.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
243382025-12-16T03:12:29.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
243392025-12-16T03:12:29.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
243402025-12-16T03:12:29.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
243412025-12-16T03:12:29.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
243422025-12-16T03:12:29.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
243432025-12-16T03:12:29.329ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
243442025-12-16T03:12:29.329ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.206494ms iteration = 161 status = {"config_updated":false}
243452025-12-16T03:12:29.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
243462025-12-16T03:12:29.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
243472025-12-16T03:12:29.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
243482025-12-16T03:12:29.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
243492025-12-16T03:12:29.345ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
243502025-12-16T03:12:29.345ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.112186ms iteration = 81 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
243512025-12-16T03:12:29.357ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
243522025-12-16T03:12:29.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
243532025-12-16T03:12:29.359ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.967421ms iteration = 81 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
243542025-12-16T03:12:31.072ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 306 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = c0802527-8dc4-493f-b427-2d7a498bba76 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
243552025-12-16T03:12:34.321ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 162 reason = Timeout
243562025-12-16T03:12:34.321ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
243572025-12-16T03:12:34.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
243582025-12-16T03:12:34.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
243592025-12-16T03:12:34.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
243602025-12-16T03:12:34.326ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
243612025-12-16T03:12:34.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
243622025-12-16T03:12:34.329ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
243632025-12-16T03:12:34.329ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.91805ms iteration = 162 status = {"config_updated":false}
243642025-12-16T03:12:39.179ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 28 reason = Timeout
243652025-12-16T03:12:39.179ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 28 reason = Timeout
243662025-12-16T03:12:39.179ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
243672025-12-16T03:12:39.179ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 28 reason = Timeout
243682025-12-16T03:12:39.179ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
243692025-12-16T03:12:39.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
243702025-12-16T03:12:39.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
243712025-12-16T03:12:39.181ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1766, tv_nsec: 787381266 } })
243722025-12-16T03:12:39.182ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1766, tv_nsec: 787899673 } })
243732025-12-16T03:12:39.182ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1350
243742025-12-16T03:12:39.182ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1766, tv_nsec: 788481500 } })
243752025-12-16T03:12:39.183ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.199673ms iteration = 28 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
243762025-12-16T03:12:39.242ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
243772025-12-16T03:12:39.244ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 65.417059ms iteration = 28 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
243782025-12-16T03:12:39.254ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1377 zone_address = fd00:1122:3344:101::2
243792025-12-16T03:12:39.254ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
243802025-12-16T03:12:39.255ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce413a28-8289-440a-99f3-dd04bc499315", "content-length": "26", "date": "Tue, 16 Dec 2025 03:12:39 GMT"} })
243812025-12-16T03:12:39.255ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1380 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
243822025-12-16T03:12:39.257ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
243832025-12-16T03:12:39.257ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1766, tv_nsec: 863382849 } })
243842025-12-16T03:12:39.295ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 116.721619ms iteration = 28 status = {}
243852025-12-16T03:12:39.319ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 28 reason = Timeout
243862025-12-16T03:12:39.319ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 55 reason = Timeout
243872025-12-16T03:12:39.320ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
243882025-12-16T03:12:39.320ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
243892025-12-16T03:12:39.320ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 71 reason = Timeout
243902025-12-16T03:12:39.320ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
243912025-12-16T03:12:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
243922025-12-16T03:12:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
243932025-12-16T03:12:39.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
243942025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 28 reason = Timeout
243952025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 28 reason = Timeout
243962025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 16.23\u{b5}s iteration = 28 status = {"disabled":true,"errors":[],"sps":[]}
243972025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
243982025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 55 reason = Timeout
243992025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 28 reason = Timeout
244002025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
244012025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 28 reason = Timeout
244022025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
244032025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 82 reason = Timeout
244042025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
244052025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
244062025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 28 reason = Timeout
244072025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
244082025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
244092025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 163 reason = Timeout
244102025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 82 reason = Timeout
244112025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
244122025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
244132025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 42 reason = Timeout
244142025-12-16T03:12:39.321ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
244152025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 28 reason = Timeout
244162025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
244172025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
244182025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 28 reason = Timeout
244192025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
244202025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 28 reason = Timeout
244212025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
244222025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 45 reason = Timeout
244232025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 28 reason = Timeout
244242025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
244252025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
244262025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 28 reason = Timeout
244272025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 28 reason = Timeout
244282025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
244292025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
244302025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 42 reason = Timeout
244312025-12-16T03:12:39.321ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
244322025-12-16T03:12:39.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
244332025-12-16T03:12:39.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
244342025-12-16T03:12:39.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
244352025-12-16T03:12:39.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
244362025-12-16T03:12:39.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
244372025-12-16T03:12:39.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
244382025-12-16T03:12:39.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
244392025-12-16T03:12:39.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
244402025-12-16T03:12:39.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
244412025-12-16T03:12:39.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
244422025-12-16T03:12:39.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
244432025-12-16T03:12:39.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
244442025-12-16T03:12:39.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
244452025-12-16T03:12:39.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
244462025-12-16T03:12:39.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
244472025-12-16T03:12:39.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
244482025-12-16T03:12:39.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
244492025-12-16T03:12:39.322ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244502025-12-16T03:12:39.323ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
244512025-12-16T03:12:39.323ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244522025-12-16T03:12:39.323ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244532025-12-16T03:12:39.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
244542025-12-16T03:12:39.324ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244552025-12-16T03:12:39.324ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244562025-12-16T03:12:39.325ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
244572025-12-16T03:12:39.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
244582025-12-16T03:12:39.326ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
244592025-12-16T03:12:39.326ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
244602025-12-16T03:12:39.326ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
244612025-12-16T03:12:39.326ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
244622025-12-16T03:12:39.326ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
244632025-12-16T03:12:39.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
244642025-12-16T03:12:39.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
244652025-12-16T03:12:39.327ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244662025-12-16T03:12:39.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
244672025-12-16T03:12:39.328ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
244682025-12-16T03:12:39.329ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
244692025-12-16T03:12:39.329ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
244702025-12-16T03:12:39.329ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
244712025-12-16T03:12:39.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
244722025-12-16T03:12:39.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
244732025-12-16T03:12:39.329ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 9.619052ms iteration = 55 status = "NoSitrep"
244742025-12-16T03:12:39.329ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
244752025-12-16T03:12:39.329ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
244762025-12-16T03:12:39.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
244772025-12-16T03:12:39.329ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
244782025-12-16T03:12:39.329ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
244792025-12-16T03:12:39.330ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 8.403087ms iteration = 28 status = {"errors":[],"requests_created_ok":[]}
244802025-12-16T03:12:39.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
244812025-12-16T03:12:39.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
244822025-12-16T03:12:39.331ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
244832025-12-16T03:12:39.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
244842025-12-16T03:12:39.332ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1766, tv_nsec: 938381329 } })
244852025-12-16T03:12:39.333ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1350 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
244862025-12-16T03:12:39.333ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 76002c35-1c9e-4f34-86ef-e4a5ebd28492 original_time_started = 2025-12-16 03:12:12.690353 UTC
244872025-12-16T03:12:39.333ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 13.438381ms iteration = 71 status = {"Loaded":{"collection_id":"76002c35-1c9e-4f34-86ef-e4a5ebd28492","time_loaded":"2025-12-16T03:12:39.319981356Z","time_started":"2025-12-16T03:12:12.690353Z"}}
244882025-12-16T03:12:39.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b1c76b61-acee-4be6-b582-973c938d6b6e (sled), lookup_type: ById(b1c76b61-acee-4be6-b582-973c938d6b6e) } result = Ok(())
244892025-12-16T03:12:39.333ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
244902025-12-16T03:12:39.333ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1766, tv_nsec: 939340471 } })
244912025-12-16T03:12:39.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
244922025-12-16T03:12:39.338ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 16.843233ms iteration = 28 status = 0
244932025-12-16T03:12:39.372ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1377 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db zone_address = fd00:1122:3344:101::2
244942025-12-16T03:12:39.372ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
244952025-12-16T03:12:39.373ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "93479fc8-0db6-41ed-a24f-9ca931255414", "content-length": "26", "date": "Tue, 16 Dec 2025 03:12:39 GMT"} })
244962025-12-16T03:12:39.373ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1380 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
244972025-12-16T03:12:39.375ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1415 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
244982025-12-16T03:12:39.378ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244992025-12-16T03:12:39.378ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245002025-12-16T03:12:39.382ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
245012025-12-16T03:12:39.382ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
245022025-12-16T03:12:39.382ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
245032025-12-16T03:12:39.382ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
245042025-12-16T03:12:39.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
245052025-12-16T03:12:39.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
245062025-12-16T03:12:39.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
245072025-12-16T03:12:39.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
245082025-12-16T03:12:39.384ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
245092025-12-16T03:12:39.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
245102025-12-16T03:12:39.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
245112025-12-16T03:12:39.384ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
245122025-12-16T03:12:39.384ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
245132025-12-16T03:12:39.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
245142025-12-16T03:12:39.389ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245152025-12-16T03:12:39.390ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245162025-12-16T03:12:39.390ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245172025-12-16T03:12:39.391ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
245182025-12-16T03:12:39.391ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 69.545161ms iteration = 163 status = {"config_updated":false}
245192025-12-16T03:12:39.391ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 69.956297ms iteration = 28 status = {"errors":[],"finish_invoked_ok":[]}
245202025-12-16T03:12:39.392ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
245212025-12-16T03:12:39.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
245222025-12-16T03:12:39.393ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245232025-12-16T03:12:39.393ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
245242025-12-16T03:12:39.393ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
245252025-12-16T03:12:39.393ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 72.116113ms iteration = 28 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
245262025-12-16T03:12:39.394ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
245272025-12-16T03:12:39.394ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
245282025-12-16T03:12:39.394ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
245292025-12-16T03:12:39.394ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
245302025-12-16T03:12:39.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
245312025-12-16T03:12:39.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
245322025-12-16T03:12:39.396ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
245332025-12-16T03:12:39.397ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
245342025-12-16T03:12:39.397ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
245352025-12-16T03:12:39.397ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245362025-12-16T03:12:39.397ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 75.527317ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
245372025-12-16T03:12:39.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
245382025-12-16T03:12:39.398ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
245392025-12-16T03:12:39.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
245402025-12-16T03:12:39.400ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
245412025-12-16T03:12:39.400ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
245422025-12-16T03:12:39.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
245432025-12-16T03:12:39.400ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
245442025-12-16T03:12:39.401ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 79.175163ms iteration = 28 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
245452025-12-16T03:12:39.401ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245462025-12-16T03:12:39.401ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 79.821842ms iteration = 28 status = {"errors":[],"garbage_collect_requested":[]}
245472025-12-16T03:12:39.402ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
245482025-12-16T03:12:39.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
245492025-12-16T03:12:39.402ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
245502025-12-16T03:12:39.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
245512025-12-16T03:12:39.403ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
245522025-12-16T03:12:39.403ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
245532025-12-16T03:12:39.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
245542025-12-16T03:12:39.404ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245552025-12-16T03:12:39.406ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
245562025-12-16T03:12:39.407ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
245572025-12-16T03:12:39.407ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 85.434312ms iteration = 55 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
245582025-12-16T03:12:39.415ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
245592025-12-16T03:12:39.415ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 94.183653ms iteration = 82 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
245602025-12-16T03:12:39.419ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1767, tv_nsec: 24874675 } })
245612025-12-16T03:12:39.423ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 101.178552ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
245622025-12-16T03:12:39.427ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
245632025-12-16T03:12:39.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
245642025-12-16T03:12:39.433ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 111.835957ms iteration = 82 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
245652025-12-16T03:12:39.441ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
245662025-12-16T03:12:39.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
245672025-12-16T03:12:39.443ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245682025-12-16T03:12:39.446ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
245692025-12-16T03:12:39.446ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
245702025-12-16T03:12:39.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
245712025-12-16T03:12:39.448ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:570
245722025-12-16T03:12:39.448ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 126.742295ms iteration = 42 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
245732025-12-16T03:12:39.453ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgd._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgd._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgd._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 4676, target: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-b1c76b61-acee-4be6-b582-973c938d6b6e.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1767, tv_nsec: 59282380 } })
245742025-12-16T03:12:39.470ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 148.816334ms iteration = 28 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
245752025-12-16T03:12:39.473ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
245762025-12-16T03:12:39.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
245772025-12-16T03:12:39.494ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
245782025-12-16T03:12:39.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
245792025-12-16T03:12:39.496ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:190 port = SwitchPort { id: 730a9ed1-b08a-4840-9edf-ffe2a2da3d0b, rack_id: b4b39465-ea7b-48fa-9a53-de5de70249db, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(3aca3fd2-36b4-4729-b4a0-744ce5c1f135) } rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
245802025-12-16T03:12:39.496ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
245812025-12-16T03:12:39.496ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
245822025-12-16T03:12:39.509ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1956 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
245832025-12-16T03:12:39.509ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1983 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 3aca3fd2-36b4-4729-b4a0-744ce5c1f135
245842025-12-16T03:12:39.509ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:428 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: Some(1) })}}
245852025-12-16T03:12:39.509ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:432 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: {V4(SwitchStaticRouteV4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, vlan: None, priority: None })}}
245862025-12-16T03:12:39.509ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:441 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
245872025-12-16T03:12:39.509ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:447 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
245882025-12-16T03:12:39.509ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:452 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
245892025-12-16T03:12:39.509ZINFOnexus (BackgroundTasks): removing static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2158 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = DeleteStaticRouteRequest { v4: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: DeleteStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
245902025-12-16T03:12:39.510ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:458 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db routes = {Switch0: AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } }}
245912025-12-16T03:12:39.510ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2203 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db request = AddStaticRouteRequest { v4: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { value: 0.0.0.0, length: 0 }, rib_priority: 1, vlan_id: None }] } }, v6: AddStaticRoute6Request { routes: StaticRoute6List { list: [] } } } switch_location = Switch0
245922025-12-16T03:12:39.510ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:467 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db
245932025-12-16T03:12:39.510ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
245942025-12-16T03:12:39.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
245952025-12-16T03:12:39.513ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:516 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch_location = Switch0
245962025-12-16T03:12:39.513ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
245972025-12-16T03:12:39.516ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
245982025-12-16T03:12:39.516ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
245992025-12-16T03:12:39.517ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "f14d4c18-bc6d-4f93-b821-12ff1825a9fd", "content-length": "372", "date": "Tue, 16 Dec 2025 03:12:39 GMT"} })
246002025-12-16T03:12:39.517ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
246012025-12-16T03:12:39.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
246022025-12-16T03:12:39.523ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
246032025-12-16T03:12:39.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
246042025-12-16T03:12:39.526ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246052025-12-16T03:12:39.528ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
246062025-12-16T03:12:39.528ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
246072025-12-16T03:12:39.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
246082025-12-16T03:12:39.537ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 215.390018ms iteration = 45 status = {"physical_disks_added":0}
246092025-12-16T03:12:39.566ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246102025-12-16T03:12:39.569ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
246112025-12-16T03:12:39.569ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
246122025-12-16T03:12:39.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
246132025-12-16T03:12:39.569ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
246142025-12-16T03:12:39.569ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
246152025-12-16T03:12:39.572ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "7d2d6963-fa07-41ff-9f5c-6449d7d968ee", "date": "Tue, 16 Dec 2025 03:12:39 GMT"} })
246162025-12-16T03:12:39.572ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
246172025-12-16T03:12:39.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
246182025-12-16T03:12:39.575ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
246192025-12-16T03:12:39.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
246202025-12-16T03:12:39.577ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
246212025-12-16T03:12:39.577ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
246222025-12-16T03:12:39.580ZINFOnexus (BackgroundTasks): VPC route manager sled b1c76b61-acee-4be6-b582-973c938d6b6e push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
246232025-12-16T03:12:39.580ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
246242025-12-16T03:12:39.586ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "4c7dc07d-cefc-490b-8c24-c439766b77a4", "date": "Tue, 16 Dec 2025 03:12:39 GMT"} })
246252025-12-16T03:12:39.588ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 267.086163ms iteration = 42 status = {}
246262025-12-16T03:12:39.603ZDEBGnexus (BackgroundTasks): client request SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
246272025-12-16T03:12:39.604ZDEBGnexus (BackgroundTasks): client response SledAgent = b1c76b61-acee-4be6-b582-973c938d6b6e background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd784ad3-63d7-45de-97b2-67d27b96e842", "content-length": "691", "date": "Tue, 16 Dec 2025 03:12:39 GMT"} })
246282025-12-16T03:12:39.604ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = buskin
246292025-12-16T03:12:39.604ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = buskin
246302025-12-16T03:12:39.604ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = buskin
246312025-12-16T03:12:39.606ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 284.597865ms iteration = 28 status = {}
246322025-12-16T03:12:39.739ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "78d53c0b-e8ef-4e34-84ed-e7b0ff7f06db", "date": "Tue, 16 Dec 2025 03:12:39 GMT"} })
246332025-12-16T03:12:39.739ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:819 rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db switch = Switch0
246342025-12-16T03:12:39.739ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
246352025-12-16T03:12:39.740ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b4b39465-ea7b-48fa-9a53-de5de70249db result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7c1885f-7c12-4d2a-9697-bfbabd0cfc6e", "content-length": "682", "date": "Tue, 16 Dec 2025 03:12:39 GMT"} })
246362025-12-16T03:12:39.740ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
246372025-12-16T03:12:39.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
246382025-12-16T03:12:39.743ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
246392025-12-16T03:12:39.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
246402025-12-16T03:12:39.747ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
246412025-12-16T03:12:39.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
246422025-12-16T03:12:39.749ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
246432025-12-16T03:12:39.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
246442025-12-16T03:12:39.757ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 437.665744ms iteration = 28 status = {}
246452025-12-16T03:12:40.362ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 5
246462025-12-16T03:12:40.362ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221
246472025-12-16T03:12:40.392ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
246482025-12-16T03:12:40.392ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:41114
246492025-12-16T03:12:40.393ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:41114 req_id = 3279be60-24c7-4459-ab7a-fd5a8c1f14b5 uri = /metrics/producers version = 1.0.0
246502025-12-16T03:12:40.393ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:41114 req_id = 3279be60-24c7-4459-ab7a-fd5a8c1f14b5 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
246512025-12-16T03:12:40.393ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:41114 req_id = 3279be60-24c7-4459-ab7a-fd5a8c1f14b5 resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
246522025-12-16T03:12:40.443ZINFOnexus: assigned collector to new producer collector_id = e5b81e4f-90b9-43ba-ba56-584fd121e601 file = nexus/src/app/oximeter.rs:76 producer_id = 76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
246532025-12-16T03:12:40.443ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 50294 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:41114 req_id = 3279be60-24c7-4459-ab7a-fd5a8c1f14b5 response_code = 201 uri = /metrics/producers
246542025-12-16T03:12:40.443ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "3279be60-24c7-4459-ab7a-fd5a8c1f14b5", "content-length": "41", "date": "Tue, 16 Dec 2025 03:12:40 GMT"} })
246552025-12-16T03:12:40.445ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
246562025-12-16T03:12:41.071ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 298 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = ec343e83-957c-4893-b9bf-f0fd36c4a7c4 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
246572025-12-16T03:12:44.320ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 164 reason = Timeout
246582025-12-16T03:12:44.321ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
246592025-12-16T03:12:44.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
246602025-12-16T03:12:44.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246612025-12-16T03:12:44.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
246622025-12-16T03:12:44.326ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
246632025-12-16T03:12:44.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
246642025-12-16T03:12:44.328ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
246652025-12-16T03:12:44.328ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.002311ms iteration = 164 status = {"config_updated":false}
246662025-12-16T03:12:49.321ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 83 reason = Timeout
246672025-12-16T03:12:49.321ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 83 reason = Timeout
246682025-12-16T03:12:49.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
246692025-12-16T03:12:49.321ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
246702025-12-16T03:12:49.321ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 165 reason = Timeout
246712025-12-16T03:12:49.321ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
246722025-12-16T03:12:49.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
246732025-12-16T03:12:49.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
246742025-12-16T03:12:49.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
246752025-12-16T03:12:49.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246762025-12-16T03:12:49.323ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
246772025-12-16T03:12:49.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
246782025-12-16T03:12:49.325ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.713437ms iteration = 83 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
246792025-12-16T03:12:49.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
246802025-12-16T03:12:49.326ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
246812025-12-16T03:12:49.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
246822025-12-16T03:12:49.328ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
246832025-12-16T03:12:49.328ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.966699ms iteration = 165 status = {"config_updated":false}
246842025-12-16T03:12:49.359ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246852025-12-16T03:12:49.362ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
246862025-12-16T03:12:49.362ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
246872025-12-16T03:12:49.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
246882025-12-16T03:12:49.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
246892025-12-16T03:12:49.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
246902025-12-16T03:12:49.366ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246912025-12-16T03:12:49.369ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
246922025-12-16T03:12:49.369ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
246932025-12-16T03:12:49.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
246942025-12-16T03:12:49.382ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
246952025-12-16T03:12:49.382ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 60.663129ms iteration = 83 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
246962025-12-16T03:12:51.071ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 270 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = bf362090-a583-46c1-8f06-6cda25fb836c response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
246972025-12-16T03:12:54.320ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 56 reason = Timeout
246982025-12-16T03:12:54.320ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
246992025-12-16T03:12:54.320ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 72 reason = Timeout
247002025-12-16T03:12:54.320ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
247012025-12-16T03:12:54.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
247022025-12-16T03:12:54.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
247032025-12-16T03:12:54.321ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 56 reason = Timeout
247042025-12-16T03:12:54.321ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 166 reason = Timeout
247052025-12-16T03:12:54.321ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
247062025-12-16T03:12:54.321ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
247072025-12-16T03:12:54.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
247082025-12-16T03:12:54.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
247092025-12-16T03:12:54.322ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
247102025-12-16T03:12:54.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
247112025-12-16T03:12:54.322ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247122025-12-16T03:12:54.323ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247132025-12-16T03:12:54.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247142025-12-16T03:12:54.324ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247152025-12-16T03:12:54.326ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
247162025-12-16T03:12:54.326ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
247172025-12-16T03:12:54.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
247182025-12-16T03:12:54.326ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
247192025-12-16T03:12:54.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
247202025-12-16T03:12:54.327ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
247212025-12-16T03:12:54.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
247222025-12-16T03:12:54.328ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 7.396684ms iteration = 56 status = "NoSitrep"
247232025-12-16T03:12:54.328ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
247242025-12-16T03:12:54.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
247252025-12-16T03:12:54.328ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
247262025-12-16T03:12:54.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
247272025-12-16T03:12:54.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
247282025-12-16T03:12:54.328ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
247292025-12-16T03:12:54.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
247302025-12-16T03:12:54.329ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 76002c35-1c9e-4f34-86ef-e4a5ebd28492 original_time_started = 2025-12-16 03:12:12.690353 UTC
247312025-12-16T03:12:54.329ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.826892ms iteration = 72 status = {"Loaded":{"collection_id":"76002c35-1c9e-4f34-86ef-e4a5ebd28492","time_loaded":"2025-12-16T03:12:54.320664522Z","time_started":"2025-12-16T03:12:12.690353Z"}}
247322025-12-16T03:12:54.330ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
247332025-12-16T03:12:54.330ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.815011ms iteration = 166 status = {"config_updated":false}
247342025-12-16T03:12:54.330ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247352025-12-16T03:12:54.331ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
247362025-12-16T03:12:54.332ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
247372025-12-16T03:12:54.332ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.750696ms iteration = 56 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
247382025-12-16T03:12:59.321ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 84 reason = Timeout
247392025-12-16T03:12:59.321ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
247402025-12-16T03:12:59.321ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 84 reason = Timeout
247412025-12-16T03:12:59.321ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 167 reason = Timeout
247422025-12-16T03:12:59.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
247432025-12-16T03:12:59.321ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
247442025-12-16T03:12:59.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
247452025-12-16T03:12:59.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
247462025-12-16T03:12:59.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
247472025-12-16T03:12:59.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247482025-12-16T03:12:59.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247492025-12-16T03:12:59.324ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
247502025-12-16T03:12:59.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
247512025-12-16T03:12:59.325ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.498686ms iteration = 84 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
247522025-12-16T03:12:59.327ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
247532025-12-16T03:12:59.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
247542025-12-16T03:12:59.328ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
247552025-12-16T03:12:59.328ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
247562025-12-16T03:12:59.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
247572025-12-16T03:12:59.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
247582025-12-16T03:12:59.329ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
247592025-12-16T03:12:59.329ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
247602025-12-16T03:12:59.329ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.948151ms iteration = 167 status = {"config_updated":false}
247612025-12-16T03:12:59.329ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
247622025-12-16T03:12:59.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247632025-12-16T03:12:59.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: a91734db-9ac7-4793-9c32-ddbb21f8d986, lookup_type: ById(a91734db-9ac7-4793-9c32-ddbb21f8d986) } result = Ok(())
247642025-12-16T03:12:59.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
247652025-12-16T03:12:59.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
247662025-12-16T03:12:59.350ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 original_time_created = 2025-12-16 02:59:03.015699 UTC target_id = a91734db-9ac7-4793-9c32-ddbb21f8d986 time_created = 2025-12-16 02:59:03.015699 UTC
247672025-12-16T03:12:59.350ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.313421ms iteration = 84 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"a91734db-9ac7-4793-9c32-ddbb21f8d986","time_created":"2025-12-16T02:59:03.015699Z"}
247682025-12-16T03:13:01.071ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:867 latency_us = 303 local_addr = [fd00:1122:3344:101::a]:54115 method = GET remote_addr = [fd00:1122:3344:101::d]:43062 req_id = deedd57f-f889-430a-a28c-fe326c866bd4 response_code = 200 uri = /76dbe3e9-13a6-4f57-a61f-1a019ca92d4c
247692025-12-16T03:13:04.321ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 168 reason = Timeout
247702025-12-16T03:13:04.321ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
247712025-12-16T03:13:04.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
247722025-12-16T03:13:04.323ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247732025-12-16T03:13:04.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
247742025-12-16T03:13:04.326ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
247752025-12-16T03:13:04.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
247762025-12-16T03:13:04.328ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false }, tuf_repo_pruner_enabled: true }, time_modified: 1970-01-01T00:01:01Z })
247772025-12-16T03:13:04.328ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.463165ms iteration = 168 status = {"config_updated":false}